Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11637 publications
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
Preview abstract
The Abkhaz-Adyghe and Nakh-Daghestanian language families encompass 35 living languages that possess arguably the most complex modern Cyrillic orthographies due to their very sophisticated phonology. The relevant online data displays idiosyncratic patterns among which the use of confusable characters in input methods is the most prevalent. This work studies one such character---letter \emph{palochka}---that is shared by most writing systems in question. We investigate whether the patterns including variants of this character alone can act as language data markers when mining these languages in a large-scale web-crawled data. Using a wide-coverage off-the-shelf LID model (GlotLID) we further investigate the data mined using such patterns and estimate the effects of confusable character normalization on quality of paragraph-level LID predictions in 14 supported languages. According to GlotLID, the normalization significantly increases the recall (discovery of new language data) for some languages while degrading it for others. However, manual evaluation reveals that only 41\% of wins and 46\% of losses are accurate due to GlotLID prediction errors. We argue that despite finding useful signal higher precision LID approaches tailored to these long-tail languages are needed to improve the quality of mined data.
View details
Bridging the Dimensionality Gap: A Taxonomy and Survey of 2D Vision Model Adaptation for 3D Analysis
Preview abstract
The remarkable success of Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) in 2D computer vision has catalyzed significant research into their adaptation for the complex domain of 3D analysis. However, a fundamental dichotomy exists between the regular, dense grid of 2D images and the irregular, sparse nature of 3D data formats such as point clouds and meshes. This paper provides a comprehensive survey and a novel intellectual framework for navigating this burgeoning field. Our core contribution is a new taxonomy that organizes adaptation strategies into three distinct families: (1) Data-centric methods, which project 3D data into 2D formats to leverage off-the-shelf 2D models; (2) Architecture-centric methods, which design intrinsic network modules to directly process 3D data; and (3) Hybrid methods, which synergistically combine pre-trained 2D features with 3D modeling processing pipelines to benefit from both rich visual priors and explicit geometric reasoning. Through this taxonomic lens, we conduct a systematic review and qualitative synthesis of the field. We illuminate the fundamental trade-offs between these families concerning computational complexity, reliance on large-scale pre-training, and the preservation of geometric inductive biases. Based on this analysis, we identify and discuss critical open challenges and chart promising future research directions, including the development of 3D foundation models, advancements in self-supervised learning for geometric data, and the deeper integration of multi-modal signals. This survey serves as an essential resource and roadmap for researchers seeking to understand and advance the state-of-the-art in 3D computer vision.
View details
Gaze Target Estimation Anywhere with Concepts
Xu Cao
Houze Yang
Vipin Gunda
Inki Kim
Jim Rehg
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2026)
Preview abstract
Estimating human gaze targets in-the-wild is a formidable challenge. Existing computer vision algorithms rely on brittle, multi-stage pipelines that require explicit inputs like head bounding boxes and human pose, causing initial detection errors to cascade and lead to system failure. To overcome this, we introduce the \textbf{Promptable Gaze Target Estimation (PGE)} task, a new end-to-end, concept-driven paradigm. PGE conditions gaze prediction on flexible user text or visual prompts (e.g., "the boy in the red shirt" or "person in point [0.52, 0.48]") to identify a specific subject's target, which eliminates the rigid dependency on intermediate localization cues. We develop a scalable data engine to generate \textbf{Gaze-Co}, a dataset and benchmark of 120K high-quality, prompt-annotated image pairs. We also propose \textbf{AnyGaze}, the first model designed for PGE. AnyGaze uses a Transformer-based detector to fuse features from frozen encoders and simultaneously solves subject localization, in/out-of-frame presence, and gaze target heatmap estimation. AnyGaze achieves state-of-the-art performance on standard gaze target estimation benchmarks, setting a strong baseline for this new problem even on a difficult out-of-domain, real-world clinical dataset. We will open-source the AnyGaze model and the Gaze-Co benchmark.
View details
Reinforcement Learning Control of Quantum Error Correction
Cameron Maxfield
Guifre Vidal
Bob Buckley
Jonathan Waltz
Christopher Wood
Reza Molavi
John Mark Kreikebaum
Rajeev Acharya
David Sobel
Abeer Vaishnav
Ali Hadjikhani
Ryuho Kudo
Wendy Leung
Brett Buchea
Ningfeng Zhu
Shirin Montazeri
Jamie Yao
Bicheng Ying
Eric Mascot
Lenny Fuste
Zhenjie Zou
Rodrigo Cortinas
Matt Lloyd
Clarke Smith
Kris Ottosson
Emma Ropes
Felix Borjans
Rebecca Potter
Sean Harrington
Jeremy Hilton
David Enriquez
Stephen Heslin
Paula Heu
Daniel Lundahl
Elliot Young
Alex Crook
Fedor Kostritsa
Roberto Rodriguez
Chia Ni
Kim Ming Lau
Priyanka Thiruraman
Martin Damyanov
Logan Oas
Dmitry Abanin
Oscar Higgott
Aaron Shorter
Steve Habegger
Aniket Maiti
Ryan Kaufman
Valerie Ehimhen
Sayra Alcaraz
Marcos Flores
Elizabeth Rossi
Aria Shahingohar
Dario Rosenstock
Travis Weidel
Steven Waltman
Kristi Wong
Murat Sarihan
Arun Kumar
Vladimir Shvarts
Matt Reagor
Alfredo Torres
Michael Qian
Anthony Megrant
Charles Neill
Christopher Hudspeth
Michael Hamilton
Bill Huggins
Laura De Lorenzo
Tan Ha
Ran Zhang
Dar Gilboa
Nicholas Bushnell
Sherman Peek
David Rhodes
Leigh Martin
Mike Shearn
Vlad Kurilovich
David Browne
Spencer Small
Brian Ballard
Will Oliver
Lior Ella
Orion Pritchard
Josh Cogan
Rachel Resnick
Dmitri Maslov
Jose Guerrero
Paul Masih Das
Theodore White
Helge Gehring
Nikita Astrakhantsev
Can Knaut
Maddy Woodson
Brooks Foxen
Frank Arute
Alejo Grajales Dau
Yaxing Zhang
Aaron Szasz
Alexander Lill
Justin Ledford
Xiaoxuan Jin
Andreas Kabel
Sid Madhuk
Orion Martin
Catherine Vollgraff Heidweiller
Gabrielle Roberts
Juan Campero
Juhwan Yoo
Robert Salazar
Michael Newman
Arpit Ranadive
James Goeders
William Giang
Gonzalo Garcia
Agnetta Cleland
Maddie Taylor
Dogan Timucin
Ross Alcaraz
Hui Kang
Johannes Bausch
William Courtney
Robert Gasca
Kevin Satzinger
Meghan Voorhees
Silas Chen
Laleh Beni
Andrew Dunsworth
Jamal Busnaina
Pavel Laptev
Kiseo Kang
Shannon Wang
Paul Donohoe
Paul Conner
Vadim Smelyanskiy
James Spencer
Benjamin Chiaro
Grayson Young
Tim Burger
ILYA Drozdov
Peter Brooks
Jordan Suchard
Austin Fowler
Jimmy Chen
Alec Eickbusch
Francisco Heras
Hung-Shen Chang
Michael Broughton
Jeanne Hartshorn
Aviv Elbag
Martin Bigdeli
Tanner Hadick
Juan Atalaya
Mahmoud Elzouka
Melvin Mathews
Alex Sztein
Markus Ansmann
Pavol Juhas
Bryan Cochrane
Murray Ich Nguyen
Ashley Maloney
Will Livingston
Roberto Collins
Ming Li
Élie Genois
Jeremiah Ford
Christopher Garrick
Sayan Das
David Peterson
Eifu Tomita
Suhas Ganjam
Reno Hiltermann
Dylan Bowers
Bryce Kobrin
Yu Chen
Dan Riley
Leon Brill
Barrett Spells
Ben Curtin
Mike Hucka
Seneca Meeks
Sebastian Molina
Tiano Lange-Dei
Georg Aigeldinger
Ashley Huff
Wing Li
ZLATKO MINEV
Monica Hansen
Sebastian Schroeder
Walt Askew
Dietrich Graumann
Elias Portoles
Stijn de Graaf
Matt Cockrell
Harold Cook
Masaya Fukami
Noah Shutty
Ed Gonzales
Robert Geiger
Amir Karamlou
Loick Le Guevel
Ebrahim Forati
Justin Vargas
Doug Thor
Joel Grebel
Lucia De Rose
LILY LI
Dave Landhuis
Emma Rosenfeld
Hsin-Yuan (Robert) Huang
Kenny Lee
Shaun Jevons
Ping Yeh
Amira Abbas
Kunal Arya
Henry Schurkus
Hector Bates
Ganesh Ramachandran
Sergey Vdovichev
Brayden Ware
Max Schaefer
Cheng Xing
Brandon Langley
Anthony Cabrera
Michel Devoret
Cody Jones
Vlad Sivak
Mert Torunbalci
Ben Kueffler
Chaitali Joshi
Raja Gosula
Joy Lee
Alexander Korotkov
Thomas Edlich
Aditya Locharla
Nathan Lacroix
George Sterling
Hao Tran
Kostyantyn Kechedzhi
Trond Andersen
Alexandre Bourassa
Aaron Lunt
Alan Fung
Alex Pizzuto
Salvatore Mandra
Alex Greene
Vitali Kutsko
Kannan Sankaragomathi
Sofia Springer
Vinicius Ferreira
Raymond Orosco
Nature (2026)
Preview abstract
The promise of fault-tolerant quantum computing is challenged by environmental drift that relentlessly degrades the quality of quantum operations. The contemporary solution, halting the entire quantum computation for recalibration, is unsustainable for the long runtimes of the future algorithms \cite{reiher2017elucidating,gidney2025factor}. We address this challenge by unifying calibration with computation, granting the quantum error correction process \cite{ryan2021realization,krinner2022realizing,sivak2023real,acharya2024quantum, bluvstein2024logical,bluvstein2025architectural,lacroix2025scaling} a dual role: its error detection events are not only used to correct the logical quantum state, but are also repurposed as a learning signal, teaching a reinforcement learning (RL) agent \cite{silver2017mastering,mnih2015human,levine2016end, shalev2016safe,ouyang2022training} to continuously steer the physical control parameters and stabilize the quantum system during the computation. We experimentally demonstrate this framework on a Willow superconducting processor, improving the logical stability of the surface code 3.5-fold against injected drift. By synthesizing our full suite of technological advances, including RL fine-tuning of the entire system and near-optimal decoding \cite{senior2025scalable, beni2025tesseract}, we achieve record performance of the surface and color codes, with average logical error per cycle of $\varepsilon_L=7.7\times10^{-4}$ and $\varepsilon_L=8.2\times10^{-3}$ respectively. Simulations of surface codes up to distance-15 with tens of thousands control parameters confirm the scalability of our RL framework, revealing an optimization speed that is independent of the system size. This work thus enables a new paradigm: a quantum computer that learns to self-improve directly from its errors and never stops computing.
View details
Preview abstract
This paper proposes RD-LoRA, a rate-distortion (R-D) optimized low-rank adaptation (LoRA) framework for neural post-filtering in the upcoming AV2 coding standard. RD-LoRA adapts a pre-trained base neural model to diverse input content via online updating of LoRA parameters, whose quantity is governed by the matrix ranks. The updated parameters are then quantized, transmitted to the decoder, and merged with the pre-trained weights for post-filtering. While more parameters generally improve coding performance, they also increase transmission bitrate.
To balance distortion reduction against transmission cost, we propose to dynamically allocate rank budget to each layer of the base model in a closed-loop R-D manner. Specifically, it incorporates a cost-aware importance assessment to discourage parameter-heavy updates, together with an RD-rank allocator to guide pruning based on global R-D optimization. To maintain robustness when certain layers are pruned to zero rank, we further introduce a lightweight fallback modulation mechanism. Experimental results show that, when deployed on a lightweight 25 kMACs ResNet model, RD-LoRA achieves a BD-rate reduction of 2.556% over the AV2 anchor, significantly outperforming the base model with only limited decoding overhead.
View details
Preview abstract
While modern text-to-image (T2I) models excel at generating high-quality images, they are typically trained to optimize with respect to generalized, population-level preferences. This homogeneous approach ignores the diverse, individual tastes and aesthetic judgments of different users. In this work, we propose a novel framework that learns fine-grained user preferences without relying on computationally expensive visual language models (VLMs) or prompt-sensitive text profiles. Instead, we introduce a robust, continuous user representation that models a user’s reward function as a linear combination of K base user types. We learn user-specific weights $\lambda_u$ via logistic regression on pairwise preference data to construct a continuous user embedding. This embedding is integrated into the diffusion process via an IP-Adapter, and fine-tuned using Diffusion-DPO. Our approach consistently generates images aligned with individual reward functions, achieving a 66.2% win rate against a pre-trained SDXL baseline and a 63.2% win rate against the state-of-the-art PPD framework.
View details
SNPeek: Side-Channel Analysis for Privacy Applications on Confidential VMs
Ruiyi Zhang
Albert Cheu
Adria Gascon
Michael Schwarz
Octavian Suciu
Network and Distributed System Security (NDSS) (2026)
Preview abstract
Confidential virtual machines (CVMs) based on trusted execution environments (TEEs) enable new privacy-preserving solutions. But CVMs are not a privacy panacea, as they are vulnerable to side-channel attacks that may compromise confidentially of workloads.
In this work, we develop the FARFETCH’D framework to help developers evaluate side-channel assisted privacy attacks that are broadly applicable to CVMs. The privacy reduction due to these attacks heavily depend on the execution environment and the workload, which varies vastly:What are avail-able attack primitives? How does the particular privacy work-load behave?This makes manual investigation and efficiently mitigating software-based side channels a cumbersome and impossible task. FARFETCH’D solves this challenge by providing a set of configurable attack primitives that can execute on real CVM hardware and automated ML-based analysis pipelines. We evaluate the effectiveness of FARFETCH’D on privacy-preserving workloads. Our results show that our approach is effective at pinpointing the vulnerability of privacy apps against side channels and help evaluating mitigation based on oblivious memory and differential privacy.
View details
A Framework for Interactive Machine Learning and Enhanced Conversational Systems
Jerry Young
Richard Abisla
Sanjay Batra
Mikki Phan
Nature, Springer-Verlag (2026)
Preview abstract
Conversational systems are increasingly prevalent, yet current versions often fail to support the full range of human speech, including variations in speed, rhythm, syntax, grammar, articulation, and resonance. This reduces their utility for individuals with dysarthria, apraxia, dysphonia, and other language and speech-related disabilities. Building on research that emphasizes the need for specialized datasets and model training tools, our study uses a scaffolded approach to understand the ideal model training and voice recording process. Our findings highlight two distinct user flows for improving model training and provide six guidelines for future conversational system-related co-design frameworks. This study offers important insights on creating more effective conversational systems by emphasizing the need to integrate interactive machine learning into training strategies.
View details
Expert evaluation of LLM world models: A high-Tc superconductivity case study
Haoyu Guo
Maria Tikhanovskaya
Paul Raccuglia
Alexey Vlaskin
Chris Co
Scott Ellsworth
Matthew Abraham
Lizzie Dorfman
Peter Armitage
Chunhan Feng
Antoine Georges
Olivier Gingras
Dominik Kiese
Steve Kivelson
Vadim Oganesyan
Brad Ramshaw
Subir Sachdev
Senthil Todadri
John Tranquada
Eun-Ah Kim
Proceedings of the National Academy of Sciences (2026)
Preview abstract
Large Language Models (LLMs) show great promise as a powerful tool for scientific literature exploration. However, their effectiveness in providing scientifically accurate and comprehensive answers to complex questions within specialized domains remains an active area of research. This work evaluates the performance of six different LLM-based systems for answering scientific literature questions, including commercially available closed models and a custom retrieval-augmented generation (RAG) system capable of retrieving images alongside text. We conduct a rigorous expert evaluation of the systems in the domain of high-temperature cuprate superconductors, a research area that involves material science, experimental physics, computation, and theoretical physics. We use an expert-curated database of 1726 scientific papers and a set of 67 expert-formulated questions. The evaluation employs a multi-faceted rubric assessing balanced perspectives, factual comprehensiveness, succinctness, evidentiary support, and image relevance. Our results demonstrate that RAG-based systems, powered by curated data and multimodal retrieval, outperform existing closed models across key metrics, particularly in providing comprehensive and well-supported answers, and in retrieving relevant visual information. This study provides valuable insights into designing and evaluating specialized scientific literature understanding systems, particularly with expert involvement, while also highlighting the importance of rich, domain-specific data in such systems.
View details
Preview abstract
Quantization methods have significantly improved the compute and memory efficiency of Large Language Model (LLM) training. However, existing approaches still rely on accumulating their updates into high precision: concretely, gradient updates must be applied to a high-precision weight buffer, known as \textit{master weights}. This buffer introduces substantial memory overhead, particularly for Sparse Mixture of Experts (SMoE) models, where model parameters and optimizer states dominate memory usage. In this work, we introduce the Error-Compensating Optimizer (ECO), which \textit{for the first time} enables the complete elimination of master weights by directly accumulating updates into quantized parameters, by leveraging existing optimizer states. ECO quantizes the weights after every gradient step and injects the resulting quantization error into the optimizer's momentum buffer, creating an error-feedback loop with zero additional memory overhead for quantization. Beyond its practical efficiency, ECO comes with theoretical guarantees. Specifically, under standard assumptions, naive master weight removal can lead to unbounded drift from the ideal parameter trajectory, whereas ECO provably bounds this drift, ensuring stable convergence. We validate ECO across a range of models, including small transformers (30M--800M), Gemma-3 1B, and an SMoE 2.1B model, using FP8 quantization. In all cases, ECO achieves near-lossless accuracy compared to high-precision baselines. For large SMoE models, ECO reduces memory usage by up to 25\%, establishing a new Pareto frontier for the trade-off between static memory and training loss.
View details
Linear-Core Surrogates: Smooth Loss Functions with Linear Rates for Classification and Structured Prediction
Proceedings of the 43rd International Conference on Machine Learning (ICML 2026)
Preview abstract
A fundamental dichotomy in the theory of classification sets smoothness against statistical efficiency: smooth surrogate losses such as the logistic loss enable fast $O(1/T)$ optimization but yield slow square-root $H$-consistency bounds, while piecewise-linear losses like the Hinge loss achieve optimal linear $H$-consistency rates but are non-differentiable. We introduce Linear-Core (LC) Surrogates, the first family of explicit convex loss functions that provably resolve this tension. By stitching a linear core to a smooth tail, we construct surrogates that are differentiable everywhere ($C^1$, and even $C^2$ under mild conditions) while retaining strict linear $H$-consistency bounds, the strongest known form of consistency guarantee. We establish these linear bounds across three increasingly complex settings: binary classification, multi-class classification, and structured prediction. To our knowledge, this is the first explicit construction to simultaneously achieve smoothness and linear $H$-consistency in any of these settings. Beyond their theoretical appeal, Linear-Core Surrogates offer practical advantages. In multi-class classification, their constant gradient profile near the decision boundary provides natural robustness to instance-dependent label noise, outperforming Cross-Entropy by 2.6% on corrupted CIFAR-10. In structured prediction, their smoothness enables an unbiased stochastic gradient estimator that bypasses the $O(|Y|^2)$ per-step complexity of exact inference, yielding a 23$\times$ speedup over Structured SVMs on large-vocabulary sequence tagging tasks.
View details
Controlling the Conversation
Martino Banchio
Bing Liu
Andres Perlroth
2026
Preview abstract
We study how AI assistants design their conversations. An assistant, with a private objective, guides a user through her search via sequential queries, modeled as partitions of the search space. The user selects the subset with the highest average value. Using a prior-free framework, we minimize the platform's regret relative to an omniscient benchmark.
We provide an optimal randomized policy that achieves bounded regret of $\frac{1}{n-1}$, where $n$ is the number of items. This policy is implemented as a sequence of ``recommendations'', that is, the AI sequentially proposes candidates from the item set until one is accepted.
View details
