Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11671 publications
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
Preview abstract
Modern transformers have been scaled up in size, data, training-time compute, and more recently test-time compute based on Chain-of-Thought (CoT), but the consequences of trading off these parameters are unknown. We construct explicit tradeoffs between two of these aspects: the number of layers in a transformer and the number of CoT tokens. Our first result is general: with mild assumptions and a single-token prompt, for any integer k, a transformer using soft attention and SwiGLU MLP layers with L layers and n CoT tokens can be converted to a transformer with at most ckL layers and ⌈n/k⌉ CoT tokens, and vice versa, although with an increase to the size of other parameters. We then a multi-token prompt path following task, finding that additional preprocessing from the multi-token prompt changes the depth-CoT relationship in this case. We construct an attention-only network with L layers and 2^(L−2) heads that can traverse 2^(L−1) − 1 edges along a path with a single CoT token, demonstrating at least an exponential relationship between increases in depth and reductions in the number of CoT tokens needed to traverse the entire path. This relationship for the path following task holds empirically as well, as demonstrated by training networks with up to 5 layers, with the tradeoff between depth and the maximum number of edge traversals in a single CoT token exactly matching our theoretical constructions.
View details
Preview abstract
The management of a hybrid workforce comprising human and autonomous computational agents may be challenged by the use of separate systems for human capital and software assets, which can create a governance gap. A system can provide a unified framework for managing a hybrid workforce. For example, the system may utilize a labor service mesh to analyze and route tasks to either a human intent tier or an agentic execution tier. A potential principle of the system is structural symmetry, where computational agents can be assigned digital identities and managed through a lifecycle process that may parallel human resource functions, such as onboarding, performance evaluation, and structured offboarding. This integrated approach can facilitate a unified system of record and governance model for an organization's intelligence capacity.
View details
Looking to the brain to improve energy efficiency of AI
Taro Toyoizumi
Hakwan Lau
Michał Klincewicz
Seng Bum Michael Yoo
Megan Peters
taylor.w.webb@gmail.com
Current Biology (2026)
Preview abstract
Modern artificial intelligence (AI) systems have achieved remarkable capabilities, but at an extraordinary energy cost. Training and running large-scale models can consume vast resources, posing environmental, economic, and social challenges. In contrast, biological brains perform lifelong learning, adaptive control, and flexible reasoning using orders of magnitude less energy for learning and adaptation over a lifetime. What accounts for this difference -- and how can it guide future AI development? In this article, we identify key biological principles that support energy-efficient capacities in biological brains, and consider how they might inform the design of more sustainable artificial systems. We organize our analysis around three domains: architectural constraints, signaling strategies, and learning algorithms. In each domain, we discuss concrete observations from biology -- from cell to circuit to cognitive level -- and describe how current and emerging AI systems mirror or diverge from these motifs. One striking feature of biological energy optimization is often overlooked: that brains are remarkably stable in their energy usage across heterogeneous modes, suggesting they may minimize energy needs during active environmental processing through maximizing the utility of “rest-like” background processes. Overall, rather than advocating for biomimicry for its own sake, we argue for biologically informed engineering. Understanding how natural systems minimize energetic cost while maximizing flexibility may help us build AI that is not only powerful, but also efficient, equitable, and environmentally responsible.
View details
StreamLens: An Open-Source ABR Simulation Environment for Multi-Region Buffering, Seeking, and Prefetching
Bei Zhao
Jinhui Song
Jiayu Zhu
Yih‑Chun Hu
2026
Preview abstract
ABR simulators are useful for studying ABR behavior under different network conditions. SABRE is an open-source simulator for evaluating ABR algorithms with different videos, network traces,
and ABR policies. However, SABRE has several limitations. First, it does not accurately emulate user seeks or their impact on the viewing experience. Second, it provides limited support for nonlinear buffering, where the client buffers non-contiguous video chunks. Furthermore, SABRE does not provide adequate analysis and visualization support for interpreting simulation results. This paper presents StreamLens, an advanced video simulator that supports: 1) pluggable ABR policies, including BOLA, BOLA-E, Dynamic, DynamicDash, and throughput-based policy; 2) configurable user seeks; 3) nonlinear buffering with customizable prefetching policy; 4) browser-based viewer for result visualization, including
stalls, seeks, rebuffering time, QoE, bitrate, buffer, etc. StreamLens allows developers to configure videos, network traces, and seek patterns, and to evaluate ABR performance across different setups, including ABR policies, linear versus nonlinear buffering, and user seek traces. It also provides metrics and visualizations that help developers inspect streaming behavior in a straightforward way. Several predefined scenarios are included as both evaluation examples and configuration templates, covering synthetic seek and prefetch settings, session-replay workflows, generated reports, and regression tests. The software is available publicly at
https://github.com/Slothbetty/StreamLens.
View details
Towards conversational artificial intelligence for disease management
Khaled Saab
David Stutz
Yong Cheng
Kavita Kulkarni
Sara Mahdavi
Joelle Barral
James Manyika
Ryutaro Tanno
Adam Rodman
Nature (2026)
Preview abstract
Although large language models have shown promise in diagnostic dialogue, their capabilities for effective management reasoning, including disease progression, therapeutic response and safe medication prescription, have remained underexplored. We have advanced the previously demonstrated diagnostic capabilities of the Articulate Medical Intelligence Explorer (AMIE) using a new large-language-model-based agentic system optimized for multivisit clinical management and dialogue. To ground the reasoning of AMIE in authoritative clinical knowledge, we leveraged the long-context capabilities of Gemini, combining in-context retrieval with structured reasoning to align its output with up-to-date clinical practice guidelines and drug formularies. In a randomized, blinded virtual Objective Structured Clinical Examination study, AMIE was compared to 21 primary care physicians (PCPs) across 100 multivisit case scenarios designed to reflect the guidance of the UK National Institute for Health and Care Excellence and BMJ Best Practice guidelines. AMIE was non-inferior to PCPs in management reasoning, as assessed by specialists, and scored better both with respect to preciseness of treatment and investigation, and in terms of its alignment with and grounding in clinical guidelines. To benchmark medication reasoning, we developed RxQA, a multiple-choice question benchmark that was derived from two national drug formularies (from the USA and UK) and validated by board-certified pharmacists. Although AMIE and PCPs both benefited from the ability to access external drug information, AMIE outperformed PCPs on higher-difficulty questions. Although further research will be needed before real-world translation of AMIE, its strong performance across evaluations marks a significant step towards use of conversational artificial intelligence as a tool in disease management.
View details
Preview abstract
In some multi-stage software build pipelines, downstream compiler errors may be reported against ephemeral, machine-generated intermediate artifacts rather than original, human-written source code, which can make remediation challenging. A system and method may address this by intercepting a downstream error, mapping its location back to the original source file, and programmatically injecting a dormant suppression tag into the original source code. During a subsequent build, an intermediate transpiler can propagate this tag into a newly generated intermediate artifact. In the intermediate file, the tag may become active and be recognized by the downstream compiler as a directive to suppress the specific error. This approach can facilitate an automated remediation process for certain build failures that avoids direct modification of ephemeral files and uses the original source code as a record for suppression.
View details
Preview abstract
We present a polynomial-time algorithm for computing an optimal committee of size $k$ under any given Thiele voting rule for elections on the Voter Interval domain (i.e., voters can be ordered so that each candidate is approved by a contiguous block of voters). Our result extends to the Generalized Thiele rule, in which each voter has an individual weight (scoring) sequence. This resolves a 10-year-old open problem that was originally posed for Proportional Approval Voting and later extended to every Thiele rule.
Our main technical ingredient is a new structural result---a concavity theorem for families of intervals. It shows that, given two solutions of different sizes, one can construct a solution of any intermediate size whose score is at least the corresponding linear interpolation of the two scores. As a consequence, on Voter Interval profiles the optimal total Thiele score is a concave function of the committee size. We exploit this concavity within an optimization framework based on a Lagrangian relaxation of a natural integer linear program formulation, obtained by moving the cardinality constraint into the objective. On Voter Interval profiles, the resulting constraint matrix is totally unimodular, so it can be solved in polynomial time.
Our main algorithm and its proof were obtained via human--AI collaboration. In particular, a slightly simplified version of the main structural theorem used by the algorithm was obtained in a single call to Gemini Deep Think.
View details
Reasoning-Driven Synthetic Data Generation and Evaluation
Tim R. Davidson
Benoit Seguin
Transactions on Machine Learning Research (2026)
Preview abstract
Although many AI applications of interest require specialized multi-modal models, relevant data to train such models is inherently scarce or inaccessible. Filling these gaps with human annotators is prohibitively expensive, error-prone, and time-consuming, leading model builders to increasingly consider synthetic data as a scalable alternative. However, existing synthetic data generation methods often rely on manual prompts, evolutionary algorithms, or extensive seed data from the target distribution — limiting their scalability, explainability, and control. In this paper, we introduce Simula: a novel reasoning-driven framework for data generation and evaluation. It employs a seedless, agentic approach to generate synthetic datasets at scale, allowing users to define desired dataset characteristics through an explainable and controllable process that enables fine-grained resource allocation. We show the efficacy of our approach on a variety of datasets, rigorously testing both intrinsic and downstream properties. Our work (1) offers guidelines for synthetic data mechanism design, (2) provides insights into generating and evaluating synthetic data at scale, and (3) unlocks new opportunities for developing and deploying AI in domains where data scarcity or privacy concerns are paramount.
View details
Inhibitory normalization of error signals improves learning in neural circuits
Roy Henha Eyono
Jonathan Cornford
Arna Ghosh
Daniel Levenstein
Neural Computation (2026)
Preview abstract
Normalization is a critical operation in neural circuits. In the brain, there is evidence that normalization is implemented via inhibitory interneurons and allows neural populations to adjust to changes in the distribution of their inputs. In artificial neural networks (ANNs), normalization is used to improve learning in tasks that involve complex input distributions. However, it is unclear whether inhibition-mediated normalization in biological neural circuits also improves learning. Here, we explore this possibility using ANNs with separate excitatory and inhibitory populations trained on an image recognition task with variable luminosity. We find that inhibition-mediated normalization does not improve learning if normalization is applied only during inference. However, when this normalization is extended to include back-propagated errors, performance improves significantly. These results suggest that if inhibition-mediated normalization improves learning in the brain, it acts primarily on learning signals rather than solely on sensory inputs.
View details
Preview abstract
Systems for escalating interactions from automated agents to human agents can create inefficiencies, for example, by transferring unstructured transcripts. An intermediary system can employ a generative artificial intelligence synthesis engine to process the context of an automated interaction upon an escalation trigger. The engine may analyze the dialogue transcript, user metadata, and the automated agent's internal state to perform semantic abstraction, diagnose potential failure points, and infer a possible resolution. The system can then generate a structured briefing for the human agent, which could include a concise summary, a failure diagnosis, or a recommended next action presented as an interactive element. This process may facilitate a more efficient handoff and contribute to an improved escalation workflow by providing the human agent with synthesized, contextual information.
View details
Preview abstract
The promise of tailored agent behavior is undermined by a critical explainability challenge: it is difficult to assess how closely and consistently the agent follows user-defined rules. As Large Language Models (LLMs) transition from static assistants to autonomous agents, developers have pioneered markdown-based rule files (e.g., GEMINI.md, CIDER_AGENT.md) to steer agent behavior and mitigate a "organizational context gap" that emerges when general-purpose models lack the "organizational context" necessary for contextually relevant results. This paper presents a qualitative study of 12 Google software developers (n=12) to investigate the authoring and efficacy of these agent rules. Our findings reveal that while rules are intended as technical steering mechanisms, they function as a "Black Box" of validation, where 12/12 participants rely on anecdotal "vibe checks" due to a profound lack of formal evaluation and explainability frameworks. We identify this opacity as a systemic Attribution Gap, which prevents developers from discerning whether a successful outcome was the result of deliberate logic or "pure luck." Paradoxically, these files serve a dual role as "Living Documentation," bridging technical instruction for AI with sociotechnical onboarding for humans. We argue for a transition toward library-level governance and rigorous traceability to transform agent customization from an ad-hoc craft into a human-centered science by revealing the internal "seams" of rule interpretation.
View details
Pixel Watch: Robust Heart Rate Sensing from Multipath PPG and On-Device Deep Learning Trained on 10,000 hours of Free-Living and Fitness Data
Megan Walker
Yojan Patel
Shyam Tailor
Matt Wimmer
Brennan Garrett
Dan Howe
Hamed Vavadi
Tien Le
Steve Diamond
Oleksiy Vyalov
Vik Sharma
Pete Richards
Tracy Giest
Erika Siegel
Tuan Phan
Sam Mravca
Derrick Vickers
Benjamin Stone
Katarina Vukosavljević
Justin Phillips
YongSuk Cho
Stefanie Hollidge
Antony Siahaan
Soren Brage
Shwetak Patel
Robert Harle
IEEE Sensors Letters (2026)
Preview abstract
The Pixel Watch 2 (PW2) is the first Google smartwatch to combine multipath photoplethysmography (PPG) with deep learning-based heart rate inference, designed to significantly improve sensing accuracy during motion-heavy activities. The device processes 10 optical channels using an on-device, 15-layer temporally dilated convolutional neural network (~300K parameters) to yield a 1 Hz heart rate output. Crucial to this model's performance was its training on a massive dataset comprising 10,000 hours of data from 962 participants, curated from a broader corpus of controlled and free-living activities. We evaluated the PW2's sensing performance across two independent validation sets: an in-house fitness dataset (229 participants, 250 hours) and an external free-living dataset (27 participants, 1000+ hours). The system achieved 95% Limits of Agreement of -10.34 to 8.66 BPM during exercise and -6.57 to 7.48 BPM during free-living activities, demonstrating substantially tighter error margins than previous Google devices. Finally, we discuss key design lessons, emphasizing that large-scale deep learning was instrumental in fully leveraging multipath PPG hardware over traditional signal processing approaches.
View details
Platform Competition in the Autobidding World
Ariel Schvartzman
Andres Perlroth
Mingfei Zhao
Proceedings of the ACM Web Conference 2026, Association for Computing Machinery, New York, NY, USA, 87–98
Preview abstract
We study the problem of auction design for advertising platforms that face strategic advertisers who are bidding across platforms. Each advertiser's goal is to maximize their total value or conversions while satisfying some constraint(s) across all the platforms they participates in. In this paper, we focus on advertisers with return-over-investment (henceforth, ROI) constraints, i.e. each advertiser is trying to maximize value while making sure that their ROI across all platforms is no less than some target value. An advertiser interacts with the platforms through autobidders -- for each platform, the advertiser strategically chooses a target ROI to report to the platform's autobidder, which in turn uses a uniform bid multiplier to bid on the advertiser's behalf on the queries owned by the given platform.
Our main result is an auction comparator theorem that can be used to choose between different auction mechanisms by a revenue-maximizing platform operating in a competitive environment. Analogous to the Linkage Principle for interdependent values, our comparator demonstrates that, for a broad class of advertiser valuations, second-price auctions may sometimes generate higher revenue than first-price auctions. This contrasts sharply with the single-platform setting, where first-price auctions are optimal for both revenue and welfare. Our findings highlight the critical importance of considering inter-platform competition when designing optimal auctions.
View details
