Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11555 publications
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
AI-PROPELLER: Warehouse-Scale Inter-Procedural Code Layout Optimization with AlphaEvolve
Chaitanya Mamatha Ananda
Rajiv Gupta
Aiden Grossman
2026
Preview abstract
Post-link optimizers (PLOs) such as Propeller and BOLT have demonstrated that precise, profile-guided code layout can extract significant performance gains from heavily optimized binaries. However, these systems are currently restricted to intra-procedural techniques, leaving the global potential of inter-procedural layout largely untapped. Inter-procedural code layout is historically difficult due to a combinatorially intractable search space and complex call-return semantics that are challenging to model. Consequently, the performance potential of fine-grained inter-procedural layout remains unproven in practice.Ours uses AlphaEvolve, an agentic workflow to evolve the compiler heuristic in Propeller into a fine-grained inter-procedural optimizer. While AlphaEvolve synthesizes novel code layout policies, Vizier fine-tunes the resulting policy hyperparameters.
To ensure high-fidelity, we move away from approximate static cost models and the agentic workflow generates multiple layout variants that are executed on actual hardware to measure real performance counters, providing a precise reward signal for the evolutionary loop. Ours has been evaluated on several benchmarks including large warehouse-scale applications and experiments show performance improvements of 0.23% to 1.6% on these benchmarks optimized with state-of-the-art FDO and PLO.
This is the first time ever that real-world applications have been optimized with fine-grained inter-procedural code layout.
View details
Preview abstract
This study examines the psychological and ethical implications of generative-AI chatbot use among youth, introducing the CTRL framework (Cognitive Trust, Reliance, and Learning Diminution) to explain how repeated use fosters cognitive offloading and reduced verification behavior. Survey data from 420 participants analyzed through factor analysis and structural equation modeling reveal that higher trust predicts greater reliance and diminished critical evaluation, alongside elevated concerns around privacy and academic integrity. Findings highlight the need for AI literacy and responsible design to mitigate unintended cognitive impacts.
View details
Mobility-Embedded POIs: Learning What a Place Is and How It’s Used from Human Movement
Shushman Choudhury
Shang-Ling Hsu
Cyrus Shahabi
Forty-third International Conference on Machine Learning (2026)
Preview abstract
Recent progress in geospatial foundation models (GeoFMs) has highlighted the
importance of learning general-purpose representations for real-world locations,
particularly Points of Interest (POIs) where human activity concentrates. Yet, ex-
isting POI representations remain largely static, drawing from textual metadata
(e.g., category labels, descriptions) and spatial attributes (e.g., coordinates, neigh-
borhood context), all of which describe what a place is, but not how it is actu-
ally used. We argue that human mobility provides a complementary and dynamic
signal, capturing real-world visitation patterns that reveal how places function in
practice. To this end, we introduce Mobility Embedded POIs (ME-POIs), a
pretraining framework that learns POI representations directly from sequences of
human visits. Each visit is encoded as a contextualized embedding that captures
the POI’s static attributes as well as its temporal and sequential context, includ-
ing when the visit occurs and which visits surround it. These visit embeddings
are aligned with learnable POI embeddings via a contrastive objective, grounding
POI representations in their real-world usage patterns. To address the long tail of
sparsely visited POIs, we transfer visitation distributions from data-rich anchors
to sparse locations, leveraging multi-scale spatial proximity to capture local and
regional patterns, and functional similarity to enable transfer across semantically
related POIs. We demonstrate the utility of ME-POIs for a set of automated map
enrichment tasks, critical in geospatial intelligence. We show empirically that
by embedding visitation dynamics, ME-POIs outperform text- and location-only
baselines, proving that mobility-informed embeddings provide a stronger founda-
tion for modeling place function and change.
View details
Preview abstract
Large language models (LLMs) are trained on web-scale corpora that exhibit steep power-law distributions, in which the distribution of knowledge is highly long-tailed, with most appearing infrequently. While scaling has improved average-case performance, persistent failures on low-frequency, domain-specific, cultural, and temporal knowledge remain poorly characterized. This paper develops a structured taxonomy and analysis of long-tail knowledge in large language models, synthesizing prior work across technical and sociotechnical perspectives.
We organize the literature along four complementary axes: how long-tail knowledge is defined, the mechanisms by which it is lost or distorted during training and inference, the technical interventions proposed to mitigate these failures, and the implications of these failures for fairness, accountability, transparency, and user trust. We further examine how existing evaluation practices obscure tail behavior and complicate accountability for rare but consequential failures. The paper concludes by identifying open challenges related to privacy, sustainability, and governance that constrain long-tail knowledge representation. Taken together, this paper provides a unifying conceptual framework for understanding how long-tail knowledge is defined, lost, evaluated, and manifested in deployed language model systems.
View details
A Framework for Interactive Machine Learning and Enhanced Conversational Systems
Jerry Young
Richard Abisla
Sanjay Batra
Mikki Phan
Nature, Springer-Verlag (2026)
Preview abstract
Conversational systems are increasingly prevalent, yet current versions often fail to support the full range of human speech, including variations in speed, rhythm, syntax, grammar, articulation, and resonance. This reduces their utility for individuals with dysarthria, apraxia, dysphonia, and other language and speech-related disabilities. Building on research that emphasizes the need for specialized datasets and model training tools, our study uses a scaffolded approach to understand the ideal model training and voice recording process. Our findings highlight two distinct user flows for improving model training and provide six guidelines for future conversational system-related co-design frameworks. This study offers important insights on creating more effective conversational systems by emphasizing the need to integrate interactive machine learning into training strategies.
View details
Preview abstract
We analyze a balls-and-bins game against an adaptive adversary that sequentially deletes bins. Starting with n balls distributed across n bins, the adversary deletes a bin in each step, forcing the algorithm to redistribute its balls to surviving bins. We prove that after n/2 rounds, uniform random redistribution yields optimal O(n) recourse and $O(\frac{\log n}{\log \log n})$ maximum load. Furthermore, we show that applying the "power of two choices"' reduces the maximum load to $O(\log \log n)$ while maintaining linear recourse.
We also consider a variation of this game where the balls from the deleted bin are partitioned evenly among d<
View details
Preview abstract
Lessons learned from building an agent to convert the R Forecast Package into a JAX api
Skander Hannachi, Jasmeet Bhatia, Dennis Kashkin, Anna Novakovska - Applied AI Engineering, Google Cloud
{shannachi@ jasmeetbhatia@ kashkin@ anovakovska@}google.com
Accepted at ISF 2026
Abstract:
The Forecast package in R, is one of the most popular and established frameworks for learning and working with statistical (local) time series models. However, over the last decade or so, the language of choice for data analysis and mathematical modeling has become Python, especially, since, unlike R, the latter provides multiple options for running models on dedicated hardware accelerators (TPUs/GPUs), and on distributed compute infrastructure.
In this study, we implement an LLM agent based code conversion pipeline that automatically converts the code from the Forecast package (R, C++), into JAX/PAX, a more recent modeling framework dedicated specifically for running compute heavy modeling tasks on TPUs and GPUs. We run our specialized code conversion agent against three core models of the Forecast package: TBATs, auto.arima(), and ETS(), with an aim to provide the exact same user experience as the original R API, but with the underlying model selection, model fitting, and forecast generation running all using JAX and PAX operations, running in a Python environment.
We report the results of our experiments and discuss the challenges we observed while running it. We compile these into a skills markdown file, which can be used by other agents intended to perform similar experiments. We provide the JAX based implementations, along with the skills file in an accompanying open source repo.
This is not the first attempt at converting the Forecast package into Python. Those other efforts however are significantly labor intensive, especially when it comes to ensuring parity with the source modeling APIs and quality control in general. Moreover, such projects rely heavily on the long term commitment of both community members and institutional contributors to the conversion effort.
The purpose of our effort is to show how this agent based process can be applied to automate any data science package upgrade or language conversion process with minimal contributors required outside of the core package maintainer team. Especially since the concept of agent skills files makes the process inherently self-improving, both within the scope of a single conversion effort, as well as across multiple long term conversion efforts. For example, the same approach can be applied to a future effort for upgrading the Forecast package to work with Julia, an even more recent and promising modeling language, while benefitting from the R-2JAX lessons learned.
View details
Preview abstract
Source-to-source compilers may perform inefficiently by executing transpilation passes on scripts that do not contain the specific language features a pass is designed to transform, potentially leading to redundant processing. A compiler can analyze a script to generate a per-script feature map, for example, by identifying language features in its abstract syntax tree (AST). Before executing a transpilation pass, the compiler can check this map and may bypass the pass for that script if the specific feature targeted by the pass is not present. This feature map can also be dynamically updated throughout the compilation process as other passes transform the code. This method of conditional pass execution based on content-aware analysis may reduce redundant AST traversals, which could decrease overall compilation time and computational resource consumption.
View details
Preview abstract
We introduce AASE (Activation-based AI Safety Enforcement), a framework for post-perception safety monitoring in large language models. Unlike pre-perception approaches that analyze input or output text, AASE monitors the model's internal activation patterns—what the model "understands" rather than what text it processes or generates—enabling detection of safety-relevant states before harmful outputs are produced. The framework comprises three techniques: Activation Fingerprinting (AF) for harmful content detection, Agent Action Gating (AAG) for prompt injection defense, and Activation Policy Compliance (APC) for enterprise policy enforcement. We introduce paired contrastive training to isolate safety-relevant signals from confounding factors such as topic and style, addressing signal entanglement in polysemantic activations. Validation across 7 models from 3 architecture families shows strong class separation: Gemma-2-9B achieves AUC 1.00 with 7.2σ separation across all probes; AAG achieves AUC ≥0.88 across all models on the InjecAgent benchmark; APC achieves 0.97-1.00 AUC across three enterprise policies. Model size correlates with probe quality—Gemma-2-9B (7.2σ separation) outperforms Gemma-2-2B (4.3σ). All techniques survive INT4 quantization with minimal separation degradation. AASE is 9× faster than Llama Guard 3 (33ms vs 306ms) with higher TPR (88% vs 50%) at a tunable threshold that trades FPR for detection sensitivity, adding only 0.002ms probe overhead to existing inference.
View details
Shuffles of Context-Free Languages along Regular Trajectories
Corentin Barloy
Michaël Cadilhac
Kyle Ockerlund
2026
Preview abstract
In single-core processors, concurrency requires that multiple processes be interleaved into a single thread of execution by a scheduler. The language-theoretic operation that corresponds to this is the shuffle of two languages: the set of words obtained by interleaving a word from each language in an arbitrary, letter-wise fashion. It is well known that regular languages are closed under shuffles, while context-free languages (CFLs) are not.
Following an established line of research, this paper considers shuffles according to regular "trajectories," that is, subject to scheduling constraints expressed by an automaton. Unsurprisingly, some trajectories allow for CFLs to be shuffled into CFLs (e.g., simple concatenation of the two words), while others do not. This paper provides a robust toolset to show that a given trajectory would always shuffle two nonregular CFLs into a nonCFL. In the case of deterministic CFLs (DCFLs), a salient trichotomy of trajectories depending on how they shuffle DCFLs is provided.
These results are based on lemmata of independent interest regarding how pushdown automata (PDA) must invoke the stack when accepting a nonregular CFL or DCFL. The latter case relies on a recent result of Jančar and Šíma (MFCS'2021); answering an open question therein, it is demonstrated that said result cannot be generalized to arbitrary CFLs, leading to dedicated machinery for both cases.
View details
Abstain and Validate: A Dual-LLM Policy for Reducing Noise in Agentic Program Repair
José Cambronero
Renyao Wei
Grant Uy
Chin-Jung Liu
Shiying Pan
Satish Chandra
ICSE-SEIP '26: Proceedings of the IEEE/ACM 48th International Conference on Software Engineering: Software Engineering in Practice, ACM (Association for Computing Machinery), New York, NY, USA (2026), 118 – 129
Preview abstract
Agentic automated program repair (APR) is increasingly being used to tackle complex, repository-level bugs in both academic and industry settings. However, changes made by the agent still need to be reviewed by a human before committing them to the codebase to ensure the change correctly fixes the bug. Showing unlikely patches to developers can lead to substantial noise, wasting valuable developer time and decreasing trust in all automated code changes. We introduce two complementary LLM-based policies to reduce such noise: bug abstention and patch critic policies. Bug abstention allows us to exclude bugs that the agentic APR system is unlikely to fix and the patch critic rejects patches that are unlikely to be a good fix for the given bug. We evaluate both policies on three sets of bugs collected from Google’s codebase and agent trajectories generated by an agentic APR system in use at Google. We introduce filtered success rates, where we compute performance after removing trajectories rejected by our policies. On a set of 174 human-reported bugs, abstention and critic can raise filtered success rates up to 13 percentage points and 15 percentage points, respectively, and up to 29 percentage points in combination. For null pointer exceptions and sanitizer-reported bugs with machine-generated bug reports or known reproduction tests, we show the same critic policy can be used to improve average single-sample success rates.
View details
MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR
Sieun Kim
Qianhui Zheng
Ruoyu Xu
Ravi Tejasvi
Anuva Kulkarni
Junyi Zhu
2026
Preview abstract
In Extended Reality (XR), complex acoustic environments often overwhelm users, compromising both scene awareness and social engagement due to entangled sound sources. We introduce MoXaRt, a real-time XR system that uses audio-visual cues to separate these sources and enable fine-grained sound interaction. MoXaRt's core is a cascaded architecture that performs coarse, audio-only separation in parallel with visual detection of sources (e.g. faces, instruments). These visual anchors then guide refinement networks to isolate individual sources, separating complex mixes of up to five concurrent sources (e.g. two voices + three instruments) with ca. 2 second processing latency. We validate MoXaRt through a technical evaluation on a new, complex dataset we collected, and a 22-participant user study. Our results demonstrate that MoXaRt significantly improves communication clarity—boosting listening comprehension in noisy conditions by 33.2% (p=0.0058)—and significantly reduces cognitive load (M=7.50 vs. M=3.36, p<0.001), paving the way for more perceptive and socially adept XR experiences.
View details
Preview abstract
This disclosure describes systems and methods for a multi-agent framework that can automate and scale cognitive work. The framework can, for example, use a cognitive assembly line of specialized computational agents to perform tasks such as research and drafting. A beneficial component could be an adversarial review panel (ARP), which is a multi-agent review system where distinct agent personas critique a generated draft from varied perspectives. The structured feedback from the ARP can be used to automatically iterate on and refine the work product. This approach can improve the intellectual rigor of generated content and reduce the time required for production, which may allow human operators to focus on activities such as strategic oversight and final validation.
View details
