Jump to Content
Mohammed Suhail

Mohammed Suhail

I am a Research Scientist with Google Research in Toronto. My research interest are in Neural Rendering and Generative Modelling. Please visit my website for more details and complete list of publications.

Research Areas

Authored Publications
Google Publications
Other Publications
results

Year

Research Area

Sort By
  • Title
  • Title, descending
  • Year
  • Year, descending
    Associating Objects and their Effects in Unconstrained Monocular Video
    Mohammed Suhail
    Erika Lu
    Zhengqi Li
    Noah Snavely
    Leonid Sigal
    Forrester Cole
    IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023
    Preview abstract We propose a method to decompose a video into a back- ground and a set of foreground layers, where the back- ground captures stationary elements while the foreground layers capture moving objects along with their associated effects (e.g. shadows and reflections). Our approach is de- signed for unconstrained monocular videos, with arbitrary camera and object motion. Prior work that tackles this problem assumes that the video can be mapped onto a fixed 2D canvas, severely limiting the possible space of camera motion. Instead, our method applies recent progress in monocular camera pose and depth estimation to create a full, RGBD video layer for the background, along with a video layer for each foreground object. To solve the under- constrained decomposition problem, we propose a new loss formulation based on multi-view consistency. We test our method on challenging videos with complex camera motion and show significant qualitative improvement over current methods. View details
    Light Field Neural Rendering
    Mohammed Suhail
    Carlos Esteves
    Leonid Sigal
    Ameesh Makadia
    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)
    Preview abstract Classical light field rendering for novel view synthesis can accurately reproduce view-dependent effects such as reflection, refraction, and translucency, but requires a dense view sampling of the scene. Methods based on geometric reconstruction need only sparse views, but cannot accurately model non-Lambertian effects. We introduce a model that combines the strengths and mitigates the limitations of these two directions. By operating on a four-dimensional representation of the light field, our model learns to represent view-dependent effects accurately. By enforcing geometric constraints during training and inference, the scene geometry is implicitly learned from a sparse set of views. Concretely, we introduce a two-stage transformer-based model that first aggregates features along epipolar lines, then aggregates features along reference views to produce the color of a target ray. Our model outperforms the state-of-the-art on multiple forward-facing and 360◦ datasets, with larger margins on scenes with severe view-dependent variations. Code and results can be found at light-field-neural- rendering.github.io. View details
    Generalizable Patch-Based Neural Rendering
    Mohammed Suhail
    Carlos Esteves
    Leonid Sigal
    Ameesh Makadia
    European Conference on Computer Vision (2022) (to appear)
    Preview abstract Neural rendering has received tremendous attention since the advent of Neural Radiance Fields (NeRF), and has pushed the state-of-the-art on novel-view synthesis considerably. The recent focus has been on models that overfit to a single scene, and the few attempts to learn models that can synthesize novel views of unseen scenes mostly consist of combining deep convolutional features with a NeRF-like model. We propose a different paradigm, where no deep visual features and no NeRF-like volume rendering are needed. Our method is capable of predicting the color of a target ray in a novel scene directly, just from a collection of patches sampled from the scene. We first leverage epipolar geometry to extract patches along the epipolar lines of each reference view. Each patch is linearly projected into a 1D feature vector and a sequence of transformers process the collection. For positional encoding, we parameterize rays as in a light field representation, with the crucial difference that the coordinates are canonicalized with respect to the target ray, which makes our method independent of the reference frame and improves generalization. We show that our approach outperforms the state-of-the-art on novel view synthesis of unseen scenes even when being trained with considerably less data than prior work. Our code is available at https://mohammedsuhail.net/gen_patch_neural_rendering. View details
    No Results Found
    Search on Google Scholar