Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11553 publications
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
Co-RedTeam: Orchestrated Security Discovery and Exploitation with LLM Agents
Pengfei He
Ash Fox
Stefan Friedli
Daniel Fabian
Burak Gokturk
Jiliang Tang
ICML 2026
Preview abstract
Large language models (LLMs) have shown promise in assisting cybersecurity tasks, yet existing approaches struggle with automatic vulnerability discovery and exploitation due to limited interaction, weak execution grounding, and a lack of experience reuse. We propose Code-RedTeam, a security-aware multi-agent framework designed to mirror real-world red-teaming workflows by integrating security-domain knowledge, code-aware analysis, execution-grounded iterative reasoning, and long-term memory. Code-RedTeam decomposes vulnerability analysis into coordinated discovery and exploitation stages, enabling agents to plan, execute, validate, and refine actions based on real execution feedback while learning from prior trajectories. Extensive evaluations on challenging security benchmarks demonstrate that Code-RedTeam consistently outperforms strong baselines across diverse backbone models, achieving over 60% attack success rate in vulnerability exploitation and up to 10% absolute improvement in vulnerability detection. Ablation and iteration studies further confirm the critical role of execution feedback, structured interaction, and memory for building robust and generalizable cybersecurity agents.
View details
PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation
Haofei Xu
Rundi Wu
Philipp Henzler
Nikolai Kalischek
Michael Oechsle
Fabian Manhardt
Marc Pollefeys
Andreas Geiger
Michael Niemeyer
International Conference on Machine Learning (ICML) (2026)
Preview abstract
Recent advances in single-image 3D reconstruction have achieved impressive fidelity. However, state-of-the-art methods often retain significant architectural overhead, relying on hybrid backbones or compressed latent representations to ensure performance. In this work, we investigate whether such complexity is strictly necessary. In particular, we introduce a pixel-space diffusion model for single-image point map prediction built upon a plain Vision Transformer (ViT). Unlike leading approaches that depend on specialized hybrid architectures or intricate loss functions, we focus on a minimalist design. We conduct a systematic investigation into the behaviors of simple architectures, identifying the key ingredients required to achieve high-fidelity dense geometry without engineering heavy-handed priors. Through controlled experiments, we demonstrate that this streamlined approach yields results superior to latent-based methods while remaining significantly simpler than hybrid alternatives. By establishing a generic, plain ViT baseline, our framework not only offers a cleaner foundation for 3D prediction but also capitalizes on the scalability of standard vision architectures, allowing it to directly benefit from future architectural advancements and extending its potential to a wider range of 3D geometry tasks.
View details
Preview abstract
In today's AI landscape, success depends not just on prompting large language models but on orchestrating them into intelligent systems that are scalable, compliant, and cost-effective. GenAI on Google Cloud is your hands-on guide to bridging that gap. Whether you're an ML engineer or an enterprise leader, this book offers a practical game plan for taking agentic systems from prototype to production.
Written by practitioners with deep experience in AgentOps, data engineering, and GenAI infrastructure, this guide takes you through real-world workflows from data prep and deployment to orchestration and integration. With concrete examples, field-tested frameworks, and honest insights, you'll learn how to build agentic systems that deliver measurable business value.
> Bridge the production gap that stalls 90% of vertical AI initiatives using systematic deployment frameworks
> Navigate AgentOps complexities through practical guidance on orchestration, evaluation, and responsible AI practices
> Build robust multimodal systems for text, images, and video using proven agent architectures
> Optimize for scale with strategies for cost management, performance tuning, and production monitoring
View details
Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AI
Jiaxun Cao
Yu Dong
Chunxi Zhan
Rithvik Neti
Pardis Emami-Naeini
USENIX Symposium on Usable Privacy and Security (SOUPS) (2026)
Preview abstract
Users increasingly rely on consumer-facing generative AI (GenAI) for tasks ranging from everyday needs to sensitive use cases. Yet, it remains unclear whether and how existing security and privacy (S&P) communications in GenAI tools shape users’ adoption decisions and experiences. Understanding how users seek, interpret, and evaluate S&P information is critical for designing usable transparency that users can trust and act on. We conducted semi-structured interviews and design sessions with 21 U.S. GenAI users. Our findings suggest that available S&P information rarely drove initial adoption in practice, as participants often perceived it as incomplete, ineffective, or not credible. Instead, they relied on rough proxies (e.g., popularity) to infer S&P practices. After adoption, S&P uncertainty constrained participants’ willingness to use GenAI tools, especially for high-stakes purposes, and, in some cases, contributed to discontinued use. Participants therefore called for transparency that supports decisions and actions through trustworthy information (e.g., independent evaluations) and usable interfaces (e.g., on-demand disclosure). We categorize participants’ desired design practices into five dimensions to facilitate systematic future investigation into best practices. We conclude with recommendations for researchers, designers, and policymakers to improve S&P transparency in consumer-facing GenAI.
View details
An Empirical Study of Tablet Ergonomics: The Interplay of Temperature, Orientation, and Use Behaviors
Carmen Van Ommen
Mikki Phan
Arun Raghupathy
Daniel Huynh
Barbara Chaparro
Ergonomics in Design: The Quarterly of Human Factors Applications Journal (2026)
Preview abstract
To balance computational performance with thermal comfort, this study explores a consolidated hotspot architecture at the top center of a tablet. We tested hotspot (39°C, 43°C, 45°C, 47°C) and ambient temperatures (25°C, 35°C) with 60 participants, measuring perception, action likelihood, and expectation. The hotspot was observed away from high contact areas, with 43°C identified as the threshold for significant discomfort. Discomfort increased with portrait mode use and higher device and ambient temperatures, while active use duration influenced acceptability. The findings underscore the importance of thermal mapping and contextual sensing, with direct applications for software throttling thresholds of coated aluminum enclosures.
View details
UniGeoCLIP: Unified Geospatial Contrastive Learning
Guillaume Astruc
Jan Hosang
Loic Landrieu
Paul-Edouard Sarlin
EarthVision Workshop at CVPR (2026)
Preview abstract
The growing availability of co-located geospatial data spanning aerial imagery, street-level views, elevation models, text, and geographic coordinates offers a unique opportunity for multimodal representation learning. We introduce UNIGEOCLIP, a massively multimodal contrastive framework to jointly align five complementary geospatial modalities in a single unified embedding space. Unlike prior approaches that fuse modalities or rely on a central pivot representation, our method performs all-to-all contrastive alignment, enabling seamless comparison, retrieval, and reasoning across arbitrary combinations of modalities. We further propose a scaled latitude-longitude encoder that improves spatial representation by capturing multi-scale geographic structure. Extensive experiments across downstream geospatial tasks demonstrate that UNIGEOCLIP consistently outperforms single-modality contrastive models and coordinate-only baselines, highlighting the benefits of holistic multimodal geospatial alignment. A reference implementation is available at https://gastruc.github.io/unigeoclip.
View details
CAST: Modeling Visual State Transitions for Consistent Video Retrieval
Yanqing Liu
Yingcheng Liu
Fanghong Dong
Budianto Budianto
Cihang Xie
Yan Jiao
2026
Preview abstract
As video content creation shifts towards long-form narratives, retrieving and composing short clips into coherent storylines becomes a critical challenge.
Standard retrieval formulations, however, perform context-agnostic retrieval, prioritizing local semantic alignment while neglecting procedural state and identity consistency across the narrative flow.
To address this, we introduce the task of Consistent Video Retrieval (CVR) and establish a benchmark designed to diagnose such inconsistencies via semantic hard negatives.
We propose CAST (Context-Aware State Transition), a lightweight adapter that models procedural progression as state-conditioned transitions.
Conditioned on visual history, CAST predicts a gated residual vector ($\Delta$) to selectively update the state embedding, ensuring procedural coherence while preserving identity.
Extensive experiments demonstrate that CAST significantly outperforms standard retrieval baselines on our CVR benchmark.
Furthermore, we show its potential as a plug-and-play consistency verifier, guiding black-box generation models (e.g., Veo) toward coherent video continuations within long-form narratives.
View details
Preview abstract
The intersection of artificial intelligence and quantitative finance is under fundamental
paradigm shift. This paper explores the transition from natural language prompting to
structured, formulaic, and programmatic methodologies. By evaluating advanced architectures such as Program of Thoughts (PoT) and Formula-One (F-1) prompting, as well as neurosymbolic integration, we demonstrate how bridging the gap between probabilistic text
generation and deterministic mathematical logic unlocks unprecedented accuracy in financial
reasoning.
View details
Preview abstract
This study examines the psychological and ethical implications of generative-AI chatbot use among youth, introducing the CTRL framework (Cognitive Trust, Reliance, and Learning Diminution) to explain how repeated use fosters cognitive offloading and reduced verification behavior. Survey data from 420 participants analyzed through factor analysis and structural equation modeling reveal that higher trust predicts greater reliance and diminished critical evaluation, alongside elevated concerns around privacy and academic integrity. Findings highlight the need for AI literacy and responsible design to mitigate unintended cognitive impacts.
View details
Preview abstract
How many T gates are needed to approximate an arbitrary n-qubit quantum state to within
a given precision ϵ? Improving prior work of Low, Kliuchnikov and Schaeffer, we show that the
optimal asymptotic scaling is Θ(sqrt{2^n log(1/ε)} + log(1/ε)) if we allow an unlimited number of ancilla qubits. We also show that this is the optimal T-count for implementing an arbitrary
diagonal n-qubit unitary to within error ϵ. We describe an application to batched synthesis of
single-qubit unitaries: we can approximate a tensor product of m = O(log log(1/ϵ)) arbitrary
single-qubit unitaries to within error ϵ with the same asymptotic T-count as is required to
approximate just one single-qubit unitary.
View details
Preview abstract
Understanding long visual documents, where information is distributed across extensive pages of text and graphics, remains a critical challenge for modern Vision-Language Models (VLMs). This difficulty is rooted in two fundamental obstacles: poor evidence localization and a high tendency for model hallucination. To address these issues, we propose DocLens, a multi-agent framework that decomposes the task into two specialized stages.
First, a Lens Module leverages document parsing tools for fine-grained, hierarchical evidence localization at both the page and element level. Second, a Reasoning Module employs a sampling-adjudication mechanism to systematically analyze the localized evidence, mitigating hallucination and synthesizing reliable answers.
Paired with Gemini-2.5-Pro, DocLens achieves state-of-the-art performance on MMLongBench-Doc and FinRAGBench-V, even surpassing human experts. Furthermore, our framework offers a highly cost-effective variant that delivers comparable performance to strong baselines at a five-fold reduction in cost.
View details
MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR
Sieun Kim
Qianhui Zheng
Ruoyu Xu
Ravi Tejasvi
Anuva Kulkarni
Junyi Zhu
2026
Preview abstract
In Extended Reality (XR), complex acoustic environments often overwhelm users, compromising both scene awareness and social engagement due to entangled sound sources. We introduce MoXaRt, a real-time XR system that uses audio-visual cues to separate these sources and enable fine-grained sound interaction. MoXaRt's core is a cascaded architecture that performs coarse, audio-only separation in parallel with visual detection of sources (e.g. faces, instruments). These visual anchors then guide refinement networks to isolate individual sources, separating complex mixes of up to five concurrent sources (e.g. two voices + three instruments) with ca. 2 second processing latency. We validate MoXaRt through a technical evaluation on a new, complex dataset we collected, and a 22-participant user study. Our results demonstrate that MoXaRt significantly improves communication clarity—boosting listening comprehension in noisy conditions by 33.2% (p=0.0058)—and significantly reduces cognitive load (M=7.50 vs. M=3.36, p<0.001), paving the way for more perceptive and socially adept XR experiences.
View details
