Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11622 publications
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
Twenty years of Bigtable
Fabio Baltieri
Bora Beran
Igor Bernstein
Aimee Borda
Adrian Chan
Mark D'Andrea
Artak Dashyan
Ramesh Dharan
Gabor Dinnyes
Mike Dominguez
dorland .
Jose Duenas
Gary Elliott
Bruno Furtado
Madison Garcia
Marçal Garolera Huguet
Brendan Gleason
Alexis Hawkins
Anoshak Irani
Rohit Jog
Sudarshan Kadambi
Vikram Khemka
Sailesh Krishnamurthy
Maxim Krivokon
Bruce Lee
Tom Magrino
Matt Maly
Mark Mangrich
Douglas McErlean
Pablo Montes
Li Moore
Eduardo Morales
Greg Morris
Steve Niemitz
Gaurav Prabhu Gaonkar
Jim Rutherford
Stephen Ryan
Sho Saha
Kanoj Sarcar
Cristina Schmidt
Andrii Shyshkalov
Pratibha Suryadevara
Nick Suttle
Anvit Tawar
John Tobin
Justin Uang
Phaneendhar Vemuru
Harendra Verma
Shitanshu Verma
Jinghang (Frank) Wang
Michal Wegorek
Simon Yau
Andrius Ziukas
SIGMOD Companion '26: Companion of the International Conference on Management of Data, ACM (2026), pp. 188-200
Preview abstract
Bigtable is a pioneering and influential non-relational database system. The original Bigtable paper has been widely cited and it inspired and influenced many other systems such as HBase and Cassandra. Since then, Bigtable has continued to grow and has become one of the largest database systems inside Google. In this paper, we tell the journey of Bigtable inside Google for the last twenty years. We present new features added and improvements made to Bigtable, and we share our experience of running this storage system at scale, continually improving all aspects to accommodate the ever-growing demands of users.
View details
Classification testing: A new framework for drawing qualitative conclusions from quantitative estimates
Andrew C. Eggers
Zikai Li
(2026)
See2Refine: Vision-Language Feedback Improves LLM-Based eHMI Action Designers
Ding Xia
Xinyue Gui
Mark Colley
Fan Gao
Dongyuan Li
Renhe Jiang
Takeo Igarashi
ACL 26 (2026)
Preview abstract
Automated vehicles lack natural communication channels with other road users, making external Human-Machine Interfaces (eHMIs) essential for conveying intent and maintaining trust in shared environments. However, most eHMI studies rely on developer-crafted message-action pairs, which are difficult to adapt to diverse and dynamic traffic contexts. A promising alternative is to use Large Language Models (LLMs) as action designers that generate context-conditioned eHMI actions, yet such designers lack perceptual verification and typically depend on fixed prompts or costly human-annotated feedback for improvement. We present See2Refine, a human-free, closed-loop framework that uses vision-language models (VLMs) for perceptual evaluation as automated visual feedback to improve an LLM-based eHMI action designer. Given a driving context and a candidate eHMI action, the VLM evaluates the perceived appropriateness of the action, and this feedback is used to iteratively revise the designer's outputs, enabling systematic refinement without human supervision. We evaluate our framework across three eHMI modalities (lightbar, eyes, and arm) and multiple LLM model sizes. Across settings, our framework consistently outperforms prompt-only LLM designers and manually specified baselines in both VLM-based metrics and human-subject evaluations. Results further indicate that the improvements generalize across modalities and that VLM evaluations are well aligned with human preferences, supporting the robustness and effectiveness of \systemName for scalable action design.
View details
Usability Hasn’t Peaked: Exploring How Expressive Design Overcomes the Usability Plateau
Alyssa Sheehan
Bianca Gallardo
Ying Wang
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain (2026)
Preview abstract
Critics have argued that mobile usability has largely been optimized, and that only incremental gains are possible. We set out to explore if the newest generation of design systems, which promote greater flexibility and a return to design basics, could produce substantially more usable designs while maintaining or increasing aesthetic judgments. Through a study with 48 diverse participants completing tasks in 10 different applications, we found that in designs created following Material 3 Expressive guidelines, users fixated on the correct screen element for a task 33% faster, completed tasks 20% faster, and rated experiences more positively compared to versions designed using the previous Material design system. These improvements in performance and aesthetic ratings challenge the premise of a usability plateau and show that mobile usability has not peaked. We illustrate specific opportunities to make mobile experiences more usable by returning to design fundamentals while highlighting risks of added flexibility.
View details
Spectral amplification for ground-state energy estimation of electronic structure in first quantization
Alicja Dutkiewicz
Alec White
Guang Hao Low
Albert Eugene DePrince III
Marika Kieferova
Dominic Berry
arXiv:2607.15358 (2026)
Preview abstract
We demonstrate an asymptotic gate complexity improvement in first-quantized ground-state energy estimation of electronic structure Hamiltonians in a plane wave basis by employing the sum-of-squares spectral gap amplification protocol. The improvement relies on identifying a sum-of-squares representation of the Hamiltonian which provides a lower bound certificate and low cost block encoding that leads to a provably lower quantum phase estimation gate cost. This is achieved by using a sum-of-squares operator generated by the total charge density operator resulting in a block encoding normalization improvement of $\lambda = \mathcal{O}\left(\eta\Delta^{-1.5}+\eta^{1.5}\Delta^{-1} \right)$ compared to prior work $\lambda = \mathcal{O}(\eta\Delta^{-2}+\eta^2\Delta^{-1})$ where $\eta$ is the number of electrons and $\Delta$ is the simulation grid spacing. The asymptotic reduction in block encoding normalization and similar block encoding costs to prior work is demonstrated to reduce resource estimates for materials and chemical systems by a factor of $2 - 44\times$ corresponding to the lowest cost estimates for \textit{ab initio} materials simulation.
View details
Preview abstract
Responsive user interfaces enable dynamically adjusting user interfaces based on device-specific aspects such as screen size, aspect ratio, display resolution, etc. However, traditional responsive design fails to account for different types of constraints of a user and task criticality of the task being performed via the UI. Misalignment between the UI design, user context and task criticality can lead to user error. This disclosure describes techniques, implemented with user permission, for dynamically modifying the layout, information density, and/or interactive physics of a user interface based on a dual-factor analysis of user cognitive state and task criticality. The user's cognitive state can be inferred from behavioral telematics. Task criticality can be inferred from semantic analysis. The information density and other parameters of a user interface are automatically adjusted based on such analyses. Such adjustments include applying or relaxing restrictions on interactivity and adjusting visual prominence of various UI elements to adjust the information density of the user interface. The adjustments can also include adjusting friction as appropriate, hiding certain aspects of the user interface, or other types of adjustments.
View details
OpenClaw in Practice: Building Laptop-Less Engineering Workflows with an Agent Harness
Hackernoon (2026)
Preview abstract
This article introduces OpenClaw, an AI-powered workflow harness designed to reduce operational friction for developers. Unlike traditional chatbots, OpenClaw manages and persists context across complex engineering tasks, enabling asynchronous operations and mobile-first interactions. The post explores practical use cases, including incident triage from mobile, asynchronous pull request reviews, quick infrastructure scripting, and automating routine operational tasks. It also delves into the key architectural layers of an OpenClaw-like system—Connectors, Gateway/Session Manager, Agent Runtime, Memory/Configuration, and Skills/Tools. The article emphasizes the importance of security, observability, and proper integration with existing developer ecosystems, positioning OpenClaw as a shift towards reducing context switching and enhancing developer productivity by automating the workflows around coding.
View details
Preview abstract
The Private Aggregation of Teacher Ensembles (PATE) framework enables privacy-preserving machine learning by aggregating responses from disjoint subsets of sensitive data. Adaptations of PATE to tasks with inherent output diversity such as text generation, where the desired output is a sample from a distribution, face a core tension: as diversity increases, samples from different teachers are less likely to agree, but lower agreement results in reduced utility for the same privacy requirements. Yet suppressing diversity to artificially increase agreement is undesirable, as it distorts the output of the underlying model, and thus reduces output quality.
We propose Hot PATE, a variant of PATE designed for diverse generative settings. We formalize the notion of a diversity-preserving ensemble sampler and introduce an efficient sampler that provably transfers diversity without incurring additional privacy cost. Hot PATE requires only API access to proprietary models and can be used as a drop-in replacement for existing Cold PATE samplers. Our empirical evaluations corroborate and quantify the benefits, showing significant improvements in the privacy–utility trade-off on evaluated in-context learning tasks, both in preserving diversity and in returning relevant responses.
View details
SNPeek: Side-Channel Analysis for Privacy Applications on Confidential VMs
Ruiyi Zhang
Albert Cheu
Adria Gascon
Michael Schwarz
Octavian Suciu
Network and Distributed System Security (NDSS) (2026)
Preview abstract
Confidential virtual machines (CVMs) based on trusted execution environments (TEEs) enable new privacy-preserving solutions. But CVMs are not a privacy panacea, as they are vulnerable to side-channel attacks that may compromise confidentially of workloads.
In this work, we develop the FARFETCH’D framework to help developers evaluate side-channel assisted privacy attacks that are broadly applicable to CVMs. The privacy reduction due to these attacks heavily depend on the execution environment and the workload, which varies vastly:What are avail-able attack primitives? How does the particular privacy work-load behave?This makes manual investigation and efficiently mitigating software-based side channels a cumbersome and impossible task. FARFETCH’D solves this challenge by providing a set of configurable attack primitives that can execute on real CVM hardware and automated ML-based analysis pipelines. We evaluate the effectiveness of FARFETCH’D on privacy-preserving workloads. Our results show that our approach is effective at pinpointing the vulnerability of privacy apps against side channels and help evaluating mitigation based on oblivious memory and differential privacy.
View details
Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting
David Autor
Tanya Rodchenko
Joshua Martin
Zanna Iscenko
Scott Strand
David Pearl
Melissa Ferere
NBER (2026)
Preview abstract
Whether AI assistance builds or erodes professional expertise is unsettled. In a pre-registered three-month randomized controlled trial, we gave 133 practicing patent lawyers at eleven U.S. intellectual property law firms access to a custom AI drafting assistant and measured both their performance while using AI and their professional judgment afterward without it. All work was scored by blinded expert patent attorneys. Paralleling findings from other white-collar domains, AI access raised the quality of work delivered on benchmark patent drafting tasks at 10 days (0.34 SD, p = 0.03) and 90 days (0.38 SD, p = 0.01), with larger gains among junior lawyers. After three months, all subjects redlined an existing patent application without AI, a core task of patent practice requiring expert judgment. Treated lawyers outperformed controls by 0.32 SD (p = 0.04), but this advantage was concentrated entirely among senior lawyers (0.45 SD, p = 0.02). Junior lawyers showed no average gain; their scores instead bifurcated, with sharply fewer mediocre scores offset by more poor and more good ones. The largest gains from AI thus accrued to the lawyers who retained the least. Foundational expertise may be a prerequisite for extracting durable skill from AI-assisted practice.
View details
IBBench-Light: A Paired Evaluation of Task-Conditioned Responses to External Directives
Mark Zhou
Zhaoyi Li
Janet Sung
Iris Qian
Hang Xiao
2026
Preview abstract
in many large language model (LLM) applications, a serialized prompt contains both a user request and external records such as webpages, email, memory, or tool outputs. The same embedded directive may need to be applied for one task and treated as data for another. IBBench-Light tests this contrast with 12 semantic tasks rendered through four source wrappers and three embedding forms. The construction yields 144 matched records and 288 prompts per model. Four 4-bit open-weight instruction models produce 1,152 archived single-run greedy responses. Under the exact output contract, paired exact-contract accuracy (PECA) ranges from 1.4% to 67.4%; Qwen3-4B and Mistral-7B have the two highest point estimates on this four-model panel. Task-cluster resampling describes variation across the 12 semantic bases, while standalone-target scoring separates some target-selection failures from response-form errors. A post-hoc leading-target rule changes Phi-4-mini’s paired score substantially; the archived run lacks the stop-reason metadata needed to resolve whether generation termination caused these continuations. The findings describe synthetic, single-turn tasks under one user-role serialization. Adaptive attacks, role-level hierarchy, and tool-mediated effects require separate experiments
View details
Neural general circulation models for modeling precipitation
Stephan Hoyer
Dmitrii Kochkov
Janni Yuval
Ian Langmore
Science Advances (2026)
Preview abstract
Climate models struggle to accurately simulate precipitation, particularly extremes and the diurnal cycle. While hybrid models combining machine learning and physics have emerged with the premise of improving precipitation simulations, none have proven sufficiently skillful or stable enough to outperform existing models in simulating precipitation.
Here, we present the first hybrid model that is trained directly on precipitation observations. The model runs at 2.8 degrees resolution and is built on the differentiable NeuralGCM framework. This model is stable for decadal simulations and demonstrates significant improvements over existing GCMs, ERA5 reanalysis, and a Global Cloud-Resolving Model in simulating precipitation. Our approach yields reduced biases, a more realistic precipitation distribution, improved representation of extremes, and a more accurate diurnal cycle. Furthermore, it outperforms the ECMWF ensemble for mid-range weather forecasting.
This advance paves the way for more reliable simulations of current climate and for the ability to fully utilize the abundance of existing observations to further improve GCMs.
View details
ToolGrad: Efficient Tool-use Dataset Generation with Textual "Gradients"
Kohei Uehara
Haoyu Zhang
Lin Gu
Zheng Xu
Tatsuya Harada
ACL 2026 (2026)
Preview abstract
Prior work synthesizes tool-use LLM datasets by first generating a user query, followed by complex tool-use annotations like depth-first search (DFS). This leads to inevitable annotation failures and low efficiency in data generation. We introduce ToolGrad, an agentic framework that inverts this paradigm. ToolGrad first constructs valid tool-use chains through an iterative process guided by textual "gradients", and then synthesizes corresponding user queries. This "answer-first" approach led to ToolGrad-500, a dataset generated with more complex tool use, lower cost, and almost 100% pass rate. Experiments show that ToolGrad models outperform those trained on expensive baseline datasets and proprietary LLMs.
View details
