Federico Tombari

Federico Tombari

Federico Tombari is a Research Director at Google, where he leads a Research&Product team on 3D Computer Vision and Multimodal ML for XR products. With his team he contributed AI technology to Google products such as Lens, Search, Maps, Android XR, Pixel phones, ARCore. He is also a Lecturer (PrivatDozent) at the Technical University of Munich (TUM). He has 300+ peer-reviewed publications in the field of 3D Computer Vision and Machine Learning and their applications to robotics, autonomous driving, healthcare and augmented/mixed reality. He obtained his PhD from the University of Bologna and his Venia Legendi (Habilitation) from Technical University of Munich (TUM). In 2018-19 he was co-founder and managing director of Pointu3D, a Munich-based startup on 3D perception for AR and robotics. He regularly serves as (Senior) Area Chair and Associate Editor for international conferences and journals (IJRR, RA-L, NeurIPS, ECCV22/24, CVPR, ICML, IROS, ICRA, 3DV among others).
Authored Publications
Sort By
  • Title
  • Title, descending
  • Year
  • Year, descending
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract 3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works. View details
PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation
Haofei Xu
Rundi Wu
Philipp Henzler
Nikolai Kalischek
Michael Oechsle
Fabian Manhardt
Marc Pollefeys
Andreas Geiger
Michael Niemeyer
International Conference on Machine Learning (ICML) (2026)
Preview abstract Recent advances in single-image 3D reconstruction have achieved impressive fidelity. However, state-of-the-art methods often retain significant architectural overhead, relying on hybrid backbones or compressed latent representations to ensure performance. In this work, we investigate whether such complexity is strictly necessary. In particular, we introduce a pixel-space diffusion model for single-image point map prediction built upon a plain Vision Transformer (ViT). Unlike leading approaches that depend on specialized hybrid architectures or intricate loss functions, we focus on a minimalist design. We conduct a systematic investigation into the behaviors of simple architectures, identifying the key ingredients required to achieve high-fidelity dense geometry without engineering heavy-handed priors. Through controlled experiments, we demonstrate that this streamlined approach yields results superior to latent-based methods while remaining significantly simpler than hybrid alternatives. By establishing a generic, plain ViT baseline, our framework not only offers a cleaner foundation for 3D prediction but also capitalizes on the scalability of standard vision architectures, allowing it to directly benefit from future architectural advancements and extending its potential to a wider range of 3D geometry tasks. View details
DRGSplat: Depth-Regularized 3D Gaussian Splatting
Daniel Barath
Keisuke Tateno
Marc Pollefeys
British Machine Vision Conference (BMVC) (2026)
Preview abstract 3D Gaussian Splatting has rapidly become a leading method for photorealistic novel view synthesis. However, its geometric accuracy often lags behind its visual fidelity. Existing methods to improve geometry typically constrain the 3D Gaussians directly, compromising their volumetric nature. We introduce DRGSplat, a novel depth-regularization approach for 3D Gaussian Splatting that enhances geometric accuracy without direct modifications of the Gaussian primitives. DRGSplat regularizes the rendered depth maps during training with three key losses: a monocular depth loss enforcing global consistency, a surface normal loss refining local detail, and a new uncertainty-aware flatness loss that selectively penalizes high-gradient regions while avoiding the gradient instabilities common to direct curvature constraints. Experiments on standard benchmarks show that DRGSplat maintains the strong photometric quality of Gaussian Splatting while substantially improving geometric accuracy and outperforming state-of-the-art geometry-focused methods. On the ETH3D dataset, DRGSplat improves reconstruction accuracy, completeness, and F1 scores by 13, 25, and 16 percentage points, respectively. The code will be made publicly available. View details
DataComp for Vision-Language Models
Matteo Farina
Vishaal Udandarao
Thao Nguyen
Selim Kuzucu
Maximilian Böther
Andreas Hochlehnert
Adhiraj Ghosh
Marianna Nezhurina
Karsten Roth
Joschka Struber
Sebastian Dziadzio
Yuhui Zhang
Elaine Sui
Soumya Jahagirdhar
Dhruba Ghosh
Hasan Hammoud
Thomas De Min
Simone Caldarella
Jehanzeb Mirza
Sedrick Keh
Mehdi Cherti
Hilde Kuehne
Bernt Schiele
Serena Yeung
Ferjad Naeem
Ana Klimovic
Elisa Ricci
Matthias Bethge
Sewoong Oh
Ameya Prabhu
Jenia Jitsev
Massimiliano Mancini
Ludwig Schmidt
Nikhil Parthasarathy
(2026)
Preview abstract Building performant Vision-Language Models (VLMs) requires carefully curating large-scale training datasets, yet the community lacks systematic benchmarks for evaluating such curation strategies. We introduce DataComp for VLMs (DCVLM), a benchmark for controlled data-centric experiments to improve VLM training. As part of DCVLM, we collect 160 datasets spanning four data types -- image-caption pairs, multimodal interleaved documents, text-only, and instruction-tuning data -- into a corpus of 6T multimodal tokens. DCVLM allows participants to test curation strategies (filtering, mixing, formatting, sampling) across 1B-8B models and 6.25B-200B token budgets. Models are then evaluated on a carefully selected suite of up to 52 downstream benchmarks across 9 domains. We conduct extensive experiments on DCVLM and find that data mixing, not filtering, is key to a high-quality training dataset: instruction-heavy mixtures scale better than caption-heavy ones, with gains widening at larger scales. The resulting dataset, DCVLM-Baseline, enables training an 8B VLM to 63.6% accuracy on our 33-task core suite with 200B training tokens. Compared to FineVision, the state-of-the-art open VLM training dataset, this represents an improvement of +5.4pp. DCVLM and all accompanying artifacts will be made publicly available at this https URL. View details
RefAM: Attention Magnets for Zero-Shot Referral Segmentation
Anna Kukleva
Enis Simsar
Ferjad Naeem
Jan Eric Lenssen
Bernt Schiele
2026
Preview abstract Internal features from large-scale pre-trained diffusion models are known to encode rich semantic information. However, standard approaches extract features by feeding noisy images into the model, degrading the quality of the representation. We propose InvertDIFT, a method that leverages diffusion inversion to recover clean representations without modifying the model. By inverting the noisy forward process to retrieve clean or intermediate states, we extract timestep-independent features from the diffusion model as is. This approach preserves the model's pre-trained knowledge and eliminates the need for additional training, while achieving performance on par or better than fine-tuned approaches across several tasks. View details
Preview abstract Reinforcement learning (RL) with verifiable rewards (RLVR) has demonstrated the great potential of enhancing the reasoning abilities in multimodal large language models (MLLMs). However, the reliance on language-centric priors and expensive manual annotations prevents MLLMs' intrinsic visual understanding and scalable reward designs. In this work, we introduce SSL-R1, a generic self-supervised RL framework that derives verifiable rewards directly from images. To this end, we revisit self-supervised learning (SSL) in visual domains and reformulate widely-used SSL tasks into a set of verifiable visual puzzles for RL-based post-training, requiring neither human nor external model supervision. Training MLLMs on these tasks substantially improves their performance on multimodal understanding and reasoning benchmarks, highlighting the potential of leveraging self-supervised vision-centric tasks for MLLM post-training. We hope this work will provide useful experience in devising effective self-supervised verifiable rewards to enable RLVR at scale. View details
Test-Time Visual In-Context Tuning
Jiahao Xie
Nathalie Rauschmayr
Bernt Schiele
2025
Preview abstract Visual in-context learning (VICL), as a new paradigm in computer vision, allows the model to rapidly adapt to various tasks with only a handful of prompts and examples. While effective, the existing VICL paradigm exhibits poor generalizability under distribution shifts. In this work, we propose test-time visual in-context tuning (VICT), a method that can learn adaptive VICL models on the fly with a single test sample. Specifically, We flip the role between task prompts and the test sample and use a cycle consistency loss to reconstruct the original task prompt output. Our key insight is that a model should be aware of a new test distribution if it can successfully recover the original task prompts. Extensive experiments on seven representative vision tasks with 15 corruptions demonstrate that our VICT can improve the generalizability of VICL to unseen new domains View details
Preview abstract Understanding fine-grained temporal dynamics is crucial in egocentric videos, where continuous streams capture frequent, close-up interactions with objects. In this work, we bring to light that current egocentric video question-answering datasets often include questions that can be answered using only few frames or commonsense reasoning, without being necessarily grounded in the actual video. Our analysis shows that state-of-the-art Multi-Modal Large Language Models (MLLMs) on these benchmarks achieve remarkably high performance using just text or a single frame as input. To address these limitations, we introduce EgoTempo, a dataset specifically designed to evaluate temporal understanding in the egocentric domain. EgoTempo emphasizes tasks that require integrating information across the entire video, ensuring that models would need to rely on temporal patterns rather than static cues or pre-existing knowledge. Extensive experiments on EgoTempo show that current MLLMs still fall short in temporal reasoning on egocentric videos, and thus we hope EgoTempo will catalyze new research in the field and inspire models that better capture the complexity of temporal dynamics. Dataset and code are available at https://github.com/google-research-datasets/egotempo.git. View details
TOKENFORMER: Rethinking Transformers Scaling with Tokenized Model Parameters
Haiyang Wang
Fan Yue
Ferjad Naeem
Jan Eric Lenssen
Liwei Wang
Bernt Schiele
2025
Preview abstract Transformers have become the predominant architecture in foundation models due to their excellent performance across various domains. However, the substantial cost of scaling these models remains a significant concern. This problem arises primarily from their dependence on fixed parameters within linear projections, especially when architectural modifications (e.g., channel dimensions) are introduced. Each scaling iteration typically requires retraining the entire model from the beginning, leading to suboptimal utilization of computational resources. To overcome this limitation, we introduce TokenFormer, a naturally scalable architecture that leverages the attention mechanism exclusively for computations among input tokens and interactions between input tokens and model parameters, thereby enhancing architectural flexibility. By treating model parameters as tokens, we replace all the linear projections in Transformer with our token-parameter attention layer, where input tokens act as queries and model parameters as keys and values. This innovative approach allows for progressive and efficient scaling without necessitating retraining from scratch. Our model scales from 124 million to 1.4 billion parameters by incrementally adding new key-value parameters, achieving performance comparable to models trained from scratch while greatly reducing training costs. Code and models will be publicly available. View details
Hierarchical 3D Scene Graphs Construction Outdoors
Jon Nyffeler
Daniel Barath
International Conference on Computer Vision (ICCV) (2025)
Preview abstract Understanding and structuring outdoor environments in 3D is critical for numerous applications, including robotics, urban planning, and autonomous navigation. In this work, we propose a pipeline to construct hierarchical 3D scene graphs from outdoor data, consisting of posed images and 3D reconstructions. Our approach systematically extracts and organizes objects and their subcomponents, enabling representations that span from entire buildings to their facades and individual windows. By leveraging geometric and semantic relationships, our method efficiently groups objects into meaningful hierarchies while ensuring robust spatial consistency. We integrate efficient feature extraction, hierarchical object merging, and relationship inference to generate structured scene graphs that capture both global and local dependencies. Our approach scales to large outdoor environments while maintaining efficiency, and we demonstrate its effectiveness on real-world datasets. We also demonstrate that these constructed outdoor scene graphs are beneficial for downstream applications, such as 3D scene alignment. View details
×