Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11597 publications
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
Preview abstract
As AI agents increasingly operate with system-level privileges across cloud environments, they face significant security vulnerabilities—most notably prompt injection and unauthorized command execution. With 72% of agent deployments experiencing security incidents within 90 days, there is a critical need for robust, defense-in-depth strategies. This article introduces "Bottom-Up AI Agent Security," a comprehensive 14-layer framework designed to secure AI agents from the foundation up. The strategy covers five essential domains: code-level security, container hardening, cloud IAM, runtime monitoring, and human oversight. By providing actionable implementation templates (including Java, YAML, and GitHub Actions) and a structured 4-week deployment roadmap, this framework enables cloud architects to transform vulnerable AI deployments into hardened, production-ready systems with measurable reductions in risk and operational costs.
View details
Preview abstract
We study advertising in conversational LLM platforms, where the platform gradually learns a user's preferences before deciding when and what ad to offer. We model this as a stochastic control problem in which the platform balances the value of information acquisition against the risk of user departure. Although the optimal policy that maximizes welfare admits a threshold structure, computing it exactly is infeasible without strong assumptions. We develop an approximate threshold policy based on upper and lower bounds on the continuation value and prove explicit performance guarantees that scale with uncertainty, learning dynamics, and heterogeneity of the advertisers. Furthermore, we extend this framework to revenue maximization, where advertisers hold private valuations. We establish a "virtual welfare equivalence" in this dynamic setting, demonstrating that the revenue-optimal incentive-compatible mechanism is implemented by applying the welfare-maximizing policy to advertisers' properly defined virtual valuations. This mechanism generalizes classical optimal auction theory to environments where the information structure evolves endogenously through user interaction, enabling us to derive approximately revenue-optimal mechanisms.
View details
An Overview of Quantization in AV2
Joe Young
Jin Zhao
In Suk Chong
Todd Nguyen
Wan-Teh Chang
Madhu Krishnan
Ryan Lei
Alican Nalci
Xiaoqing Zhu
Joel Sole
Hilmi Eglimez
Luhang Xu
Yue Yu
Dong Wang
(2026)
Preview abstract
This paper presents an overview of the quantization techniques used in the AV2 video coding standard. For scalar quantization, an improved design consolidates six lookup tables into a simplified function where the step size doubles with every 24 quant index increments. Support for user-defined quantization matrices can be signaled in the sequence header, enabling segment-level matrix selection. Moreover, advanced quantization methods are described, including a parity-hiding method which can implicitly signal the parity of the DC (index 0,0) coefficient, based on the sum of the AC coefficients, thereby reducing the number of explicitly coded symbols. Additionally, an eight-state trellis-coded quantization method is introduced as a high-performance alternative to scalar quantization. This paper covers the rationale, implementation details, and the resulting coding gain benefits of these methods.
View details
Climate change shifts the North Pacific storm track polewards
Rei Chemke
Janni Yuval
Nature (2026)
Preview abstract
Across the North Pacific Ocean, the mid-latitude storm track accounts for most of the heat and moisture transport into the Arctic and western North America, considerably influencing regional precipitation and temperature patterns¹, ². By the end of this century, the winter North Pacific storm track is projected to shift poleward³⁻⁶, with substantial implications for oceanic ecosystems and land-based water availability¹, ⁷. While atmospheric reanalyses suggest a poleward shift of the storm track⁷⁻¹³, the lack of an observed wind record has left it uncertain whether the storm track shift has occurred in recent decades, and what role climate change plays in determining the storm track position. Here, we derive an observational constraint for mid-latitude storm tracks and show that the winter North Pacific storm track has shifted substantially poleward, emerging from natural variability. A poleward shift of storm track-induced heat and moisture flux is also evident over western North America, implying regional impacts on precipitation and temperature patterns. Our analysis further reveals that climate models underestimate the poleward shift of the storm track in recent decades, suggesting that the future human-induced impacts on both the North Pacific ecosystem and western North America might be larger than in current predictions.
View details
Bridging the Dimensionality Gap: A Taxonomy and Survey of 2D Vision Model Adaptation for 3D Analysis
Preview abstract
The remarkable success of Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) in 2D computer vision has catalyzed significant research into their adaptation for the complex domain of 3D analysis. However, a fundamental dichotomy exists between the regular, dense grid of 2D images and the irregular, sparse nature of 3D data formats such as point clouds and meshes. This paper provides a comprehensive survey and a novel intellectual framework for navigating this burgeoning field. Our core contribution is a new taxonomy that organizes adaptation strategies into three distinct families: (1) Data-centric methods, which project 3D data into 2D formats to leverage off-the-shelf 2D models; (2) Architecture-centric methods, which design intrinsic network modules to directly process 3D data; and (3) Hybrid methods, which synergistically combine pre-trained 2D features with 3D modeling processing pipelines to benefit from both rich visual priors and explicit geometric reasoning. Through this taxonomic lens, we conduct a systematic review and qualitative synthesis of the field. We illuminate the fundamental trade-offs between these families concerning computational complexity, reliance on large-scale pre-training, and the preservation of geometric inductive biases. Based on this analysis, we identify and discuss critical open challenges and chart promising future research directions, including the development of 3D foundation models, advancements in self-supervised learning for geometric data, and the deeper integration of multi-modal signals. This survey serves as an essential resource and roadmap for researchers seeking to understand and advance the state-of-the-art in 3D computer vision.
View details
Accelerating Post-Silicon Validation : ATE Bring-up and Test Leveraging Tessent Streaming Scan Network (SSN) for High Bandwidth IJTAG
Gaurav Mattey
Rakesh Kinger
Michael Reese
Bruno Marques da Costa
2026
Preview abstract
As SoC complexity grows, traditional serial JTAG/IJTAG interfaces increasingly become a bottleneck for post-silicon validation and instrument orchestration. This paper details the first-pass ATE bring-up and test experience using Tessent High-Bandwidth IJTAG (HB-IJTAG), a solution that maps IEEE 1687 control sequences directly onto the Streaming Scan Network (SSN) bus fabric
View details
Preview abstract
Mid-air gestures in Extended Reality (XR) often lead to fatigue, discomfort and imprecision, limiting their suitability for extended use. Surface-based interactions offer a compelling alternative, providing improved accuracy, speed, and comfort. However, current egocentric vision-based methods struggle with reliable surface inputs due to challenges in hand tracking and surface-plane estimation from oblique and occluded viewing angles. To this extent, we introduce SurfaceXR, a novel sensor fusion approach that combines headset based hand tracking with micro-vibration data sampled from commodity smartwatch IMUs to enable precise and robust inputs on arbitrary surfaces. Our system is designed with flexibility in mind - it can function using only hand tracking, only IMU sensing, or optimally with both modalities combined. Our user study across 12 participants validates SurfaceXR's effectiveness in augmenting surface touch tracking and 8 class hand-surface gesture recognition, demonstrating significant improvements over single-modality approaches. Enabled by SurfaceXR, we demonstrate a series of interactive apps for both AR and VR, ranging from on-surface sketching, text entry and gesture based navigation.
View details
Preview abstract
High-fidelity removal of eyeglasses from video is a major challenge in facial attribute editing, as the underlying facial geometry is often obscured by complex refractive distortions and view-dependent specular reflections. While large-scale generative priors have shown promise in static image inpainting, they often lack the structural constraints necessary to maintain identity, expression, and pose, leading to visible “identity drift” in both static images and dynamic sequences. In this paper, we propose a novel distillation framework that addresses the stochastic nature of generative priors. Our pipeline first extracts high-fidelity outputs from Gemini, regularizes them via dense geometric landmark constraints to preserve identity, expression, and pose, and finally applies physically-based simulation of lens optics to generate realistic paired training data. This process transfers Gemini’s photo-realistic, multi-view knowledge into a specialized restoration architecture, JFSnet (Joint Feature-Spatial network). JFSnet integrates DINOv2-based semantic features with a convolutional decoder for spatial reconstruction, leveraging equivariance constraints to improve high-frequency detail preservation and stability. Evaluations show that our approach significantly outperforms existing diffusion and GAN-based baselines, achieving the lowest FID scores and ranking highest in user preference studies across visual fidelity, identity preservation, and temporal consistency.
View details
Integrating Level-Design for Editable, Shared Worlds in Diffusion Game Engines
Po Ryan
David Zhang
Amir Hertz
Gordon Wetzstein
Neal Wadhwa
Nataniel Ruiz
2026
Preview abstract
Video world models have shown immense promise for interactive simulation and entertainment, but current systems still struggle with two important aspects of interactivity: user control over the environment for reproducible, editable experiences, and shared inference where players hold influence over a common world. To address these limitations, we introduce an explicit external memory into the system, a persistent state operating independent of the model’s context window, that is continually updated by user actions and queried throughout the generation roll-out. Unlike conventional diffusion game engines that op-
erate as next-frame predictors, our approach decomposes generation into Memory, Observation, and Dynamics modules. This design gives users direct, editable control over environment structure via an editable memory representation, and it naturally extends to real-time multiplayer rollouts with coherent viewpoints and consistent cross-player interactions.
View details
Preview abstract
Existing image simplification techniques often rely on Non-Photorealistic Rendering (NPR), transforming photographs into stylized sketches, cartoons, or paintings. While effective at reducing visual complexity, such approaches typically sacrifice photographic realism. In this work, we explore a complementary direction: simplifying images while preserving their photorealistic appearance.
We introduce progressive semantic image simplification, a framework that iteratively reduces scene complexity by removing and inpainting elements in a controlled manner. At each step, the resulting image remains a plausible natural photograph. Our method combines semantic understanding with generative editing, leveraging Vision-Language Models (VLMs) to identify and prioritize elements for removal, and a learned verifier to ensure photorealism and coherence throughout the process. This is implemented via an iterative \emph{Select–Remove–Verify} pipeline that produces high-quality simplification trajectories.
To improve efficiency, we further distill this process into an image-to-video generation model that directly predicts coherent simplification sequences from a single input image. Beyond generating cleaner and more focused compositions, our approach enables applications such as content-aware decluttering, semantic layer decomposition, and interactive editing.
More broadly, our work suggests that simplification through structured content removal can serve as a practical mechanism for guiding visual interpretation within the photorealistic domain, complementing traditional abstraction methods.
View details
Platform Competition in the Autobidding World
Ariel Schvartzman
Andres Perlroth
Mingfei Zhao
Proceedings of the ACM Web Conference 2026, Association for Computing Machinery, New York, NY, USA, 87–98
Preview abstract
We study the problem of auction design for advertising platforms that face strategic advertisers who are bidding across platforms. Each advertiser's goal is to maximize their total value or conversions while satisfying some constraint(s) across all the platforms they participates in. In this paper, we focus on advertisers with return-over-investment (henceforth, ROI) constraints, i.e. each advertiser is trying to maximize value while making sure that their ROI across all platforms is no less than some target value. An advertiser interacts with the platforms through autobidders -- for each platform, the advertiser strategically chooses a target ROI to report to the platform's autobidder, which in turn uses a uniform bid multiplier to bid on the advertiser's behalf on the queries owned by the given platform.
Our main result is an auction comparator theorem that can be used to choose between different auction mechanisms by a revenue-maximizing platform operating in a competitive environment. Analogous to the Linkage Principle for interdependent values, our comparator demonstrates that, for a broad class of advertiser valuations, second-price auctions may sometimes generate higher revenue than first-price auctions. This contrasts sharply with the single-platform setting, where first-price auctions are optimal for both revenue and welfare. Our findings highlight the critical importance of considering inter-platform competition when designing optimal auctions.
View details
Preview abstract
How many T gates are needed to approximate an arbitrary n-qubit quantum state to within
a given precision ϵ? Improving prior work of Low, Kliuchnikov and Schaeffer, we show that the
optimal asymptotic scaling is Θ(sqrt{2^n log(1/ε)} + log(1/ε)) if we allow an unlimited number of ancilla qubits. We also show that this is the optimal T-count for implementing an arbitrary
diagonal n-qubit unitary to within error ϵ. We describe an application to batched synthesis of
single-qubit unitaries: we can approximate a tensor product of m = O(log log(1/ϵ)) arbitrary
single-qubit unitaries to within error ϵ with the same asymptotic T-count as is required to
approximate just one single-qubit unitary.
View details
Preview abstract
For many practical applications of quantum computing, the slowest and most costly steps involve coherently accessing classical data. We help address this challenge by applying mass production techniques, which can sometimes allow us to perform operations many times in parallel for a cost that is comparable to a single execution[1-3]. We combine existing mass-production results with modern approaches for loading classical data using ``quantum read-only memory.'' We show that quantum mass production techniques offer no benefit when we consider a cost model that focuses purely on the number of non-Clifford gates. However, analyzing the constant factors in a more nuanced cost model, we find that it may be possible to obtain a reduction in cost of an order or magnitude or more for a variety reasonably-sized fault-tolerant quantum algorithms. We present several applications of quantum mass-production techniques beyond naive parallelization, including a strategy for reducing the cost of serial calls to the same data loading step.
View details
