Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11635 publications
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
Preview abstract
Distributed federated SQL query engines frequently
materialize query results into files in data lakes. Optimizing the
sizes of these files (e.g., balancing their sizes) is crucial for the
efficiency of not only the materialization queries themselves, but
also subsequent queries that read these files. Existing techniques
to manage file size include specifying output targets (e.g., number
of partitions), using cardinality estimation for materialized data
volume, performing full shuffles prior to materialization to obtain
accurate statistics, and/or post (background) compactions to
merge small files. All of these solutions have limitations in
practice; they typically do not provide any strong guarantees or
can be prohibitively expensive when they do (e.g., they require
background compactions to merge small files, which requires
additional I/O and CPU cost). Most of the existing techniques
can produce many small files during materialization and are
sensitive to data skew.
In this paper, we propose a novel method that streamlines
materialization within the same query execution and provides
guarantees on the file sizes using statistics at runtime. Our
approach does not require a full shuffle before the materialization
operator in order to get accurate statistics, making it attractive
and robust in practice. To show the effectiveness of our approach,
we present production metrics from F1 Query at Google that has
been running this functionality for the majority of its production
workload over many quarters. Our approach reduces the number
of files produced by a factor of 100 or more, and as a result,
it avoids creating tens of billions of files per day, saving storage
and computation cost.
View details
Approximate vs Precise: An experiment in what impacts user choice when apps request location access
Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26), April 13–17, 2026, Barcelona, Spain (2026)
Preview abstract
User location data is highly sensitive, yet commonly requested by mobile apps for both core functionality and monetization. To improve user privacy, the major mobile platforms, Android and iOS, made changes so that when apps request precise location access, users can choose to share only their approximate location. However, the platforms have diverging interfaces: Android offers a side-by-side choice and iOS offers a corner toggle. This study evaluates which factors impact users’ choices when apps request location access via a randomized controlled experiment with 2579 US Android users. We tested the impact of app type, whether a reason for the request was provided, and the quality and content of the reason, including monetization. We do not find the reasons have an effect. Instead, we find users’ choices are impacted by app type and user demographics. We find that when users are given a side-by-side choice to allow approximate versus precise location access, they make reasonable choices. Of users who allowed access, the vast majority (90.7%) chose precise for a rideshare app versus the majority (71.3%) chose approximate for a local news app. Concerningly, the majority also allowed location access to a wallpaper app, and older users were significantly more likely to allow apps precise location access. We conclude by discussing implications for app platforms and future work.
View details
Prospective multi-pathogen disease forecasting using autonomous LLM-guided tree search
Sarah Martinson
Nicholas Reich
Zahra Shamsi
arXiv (2026)
Preview abstract
Probabilistic forecasting of infectious diseases is crucial for public health but relies on labor-intensive manual curation by expert modeling teams. This bespoke development bottlenecks scalability to granular geographic resolutions or emerging pathogens. Here, we present an autonomous system utilizing Large Language Model (LLM)-guided tree search \cite{aygun_ai_2025} to iteratively generate, evaluate, and optimize executable forecasting software. In a fully prospective, real-time evaluation during the 2025–2026 US respiratory season, the system autonomously discovered methodologically diverse models for influenza, COVID-19, and respiratory syncytial virus (RSV). Aggregating these machine-generated models yielded an ensemble that consistently matched or outperformed the gold-standard, human-curated Centers for Disease Control and Prevention (CDC) hub ensembles out-of-sample. The system successfully navigated data-scarce "cold start" scenarios for RSV. Moreover, controlled ablations revealed that optimizing log-scale distance metrics prevents reward hacking, while an automated judge-in-the-loop ensures structural fidelity to complex scientific theories. By autonomously translating epidemiological theory into accurate, transparent code, this framework breaks the modeling labor bottleneck, enabling the rapid deployment of expert-level disease forecasting at unprecedented scales.
View details
Preview abstract
We study the problem of incentivizing strategic agents to truthfully contribute high-quality data in collaborative learning settings, where each agent benefits from improved estimation based on others’ data. Each agent privately observes the quality of their data, and agents may misreport it if not incentivized properly. We cast this problem with a Bayesian mechanism design framework in which the platform aims to find the optimal data-sharing mechanism that jointly determines allocations and payments to maximize both estimation accuracy and platform revenue. We prove that the optimal mechanism that incentivizes truthful reporting takes the form of a \emph{personalized threshold and pricing} mechanism, in which each agent is allocated the learned estimator if their reported quality exceeds a (personalized) threshold and is charged a price based on the relevance of other agents' data in the learning task. We analyze this mechanism in a canonical Gaussian mean estimation task, derive a closed-form solution to the optimal mechanism, and highlight how data correlation affects the mechanism. We further extend the model to allow agents to exert costly efforts to improve their data quality before collaboration. We show that ''free-riding'' is mitigated as the optimal data-sharing mechanism induces a supermodular game: each agent is incentivized to exert more effort when others exert more.
Finally, we show that equilibrium efforts form a complete lattice, and in the highest-effort equilibrium, each agent increases effort as others' data becomes more relevant in the learning task.
View details
Preview abstract
Summing vectors is a basic task in differentially private data analysis. Standard algorithms clip vectors to some bound, sum them, and add noise scaled to the bound. While the differential privacy literature has developed a deep library of such noise addition mechanisms, it offers few tools for privately choosing the clipping bound directly from the data. We introduce a method that privately chooses a clipping bound by optimizing \emph{residual coherence}, the directional alignment of vectors exceeding the bound, against the variance cost of additional noise. We prove a close relationship between residual coherence and bias for a general class of heavy-tailed data regimes and show empirically that, across several datasets, our method outperforms existing baselines.
View details
Preview abstract
Chain-of-Thought (CoT) improves LLM reasoning but amplifies decoding latency and memory usage. While latent reasoning attempts to shift this computation to internal hidden states, it is hindered by the parallel nature of the Transformer prefill—which blocks sequential deep-to-shallow information flow—and the high training costs of recurrent optimization. We propose a supervised latent reasoning approach that enables efficient sequential computation during prefill. Our method augments an LLM with a recurrent pathway to propagate intermediate "thinking" tokens from deep to shallow layers without expanding the KV-cache. Unlike prior unsupervised methods, we utilize structured supervision to train this mechanism via teacher forcing, eliminating the need for Backpropagation Through Time (BPTT). Evaluations on state-tracking and reasoning benchmarks demonstrate that our approach outperforms existing latent baselines and approaches CoT performance, effectively combining the reasoning power of CoT with the inference efficiency of standard models.
View details
Preview abstract
Large language models (LLMs) are trained on web-scale corpora that exhibit steep power-law distributions, in which the distribution of knowledge is highly long-tailed, with most appearing infrequently. While scaling has improved average-case performance, persistent failures on low-frequency, domain-specific, cultural, and temporal knowledge remain poorly characterized. This paper develops a structured taxonomy and analysis of long-tail knowledge in large language models, synthesizing prior work across technical and sociotechnical perspectives.
We organize the literature along four complementary axes: how long-tail knowledge is defined, the mechanisms by which it is lost or distorted during training and inference, the technical interventions proposed to mitigate these failures, and the implications of these failures for fairness, accountability, transparency, and user trust. We further examine how existing evaluation practices obscure tail behavior and complicate accountability for rare but consequential failures. The paper concludes by identifying open challenges related to privacy, sustainability, and governance that constrain long-tail knowledge representation. Taken together, this paper provides a unifying conceptual framework for understanding how long-tail knowledge is defined, lost, evaluated, and manifested in deployed language model systems.
View details
Preview abstract
Multimodal large language models (LLMs) integrate and process information from multiple modalities such as text, images, audio, and video, enabling complex tasks such as audio translation and visual question answering. While powerful, this complexity introduces novel vulnerabilities to sophisticated adversarial attacks. This survey paper provides a comprehensive overview of this rapidly expanding field, systematically categorizing attacks that range from manipulations of single modalities (e.g., perturbed images or audio) to those exploiting cross-modal interactions. We overview how these attacks exploit weaknesses in model fusion, attention mechanisms, and representation learning and provided analyses on their potential for real-world consequences.
View details
Preview abstract
Forensic verification often uses a binary “real vs. fake” label that groups fully synthetic, tampered, and AI-retouched images despite their different consequences. We study these modifications through two complementary channels: a camera channel sensitive to capture and processing traces, and a semantic channel capturing scene content. The channels provide continuous evidence rather than deterministic signatures of manipulation history. We instantiate this perspective in 2CAP (2-Channel Authenticity Protocol), pairing a contrastively trained camera encoder with a frozen semantic encoder for (i) reference-free four-class classification through reliability-weighted cross-attention fusion and (ii) reference-based evidence generation. For the latter, query-reference channel similarities and a patch-level saliency map guide a frozen Vision Language Model (VLM) through an Observe–Generate–Refine loop, without forensic instruction tuning of the VLM. On the evaluated benchmark, 2CAP achieves overall AUC .963 and F1 .844; retouching F1 improves from .868 for the strongest compared baseline to .961. Across five VLM configurations, shared-parser paired evaluation shows model-dependent effects: evidence improves change-type accuracy over evidence-free refinement. These results support manipulation-type discrimination while delimiting the benefits of frozen-VLM refinement.
View details
Mobile Telemetry-Centered Threat Modeling for Managed Enterprise Architectures
IEEE Computer, vol. 59 (2026)
Preview abstract
This work introduces Telemetry-Centered Threat Modeling (TCTM), a vendor- and operating system–agnostic framework designed to assess whether mobile security telemetry is sufficiently authoritative to guide enterprise security policies and automated decision-making. Recognizing that enterprise security workflows frequently falter when acting on intercepted, delayed, or manipulated signals even if endpoint enforcement remains functional, TCTM establishes telemetry as a first-class security surface rather than a passive byproduct. The framework systematically evaluates evidence paths across four essential properties (origin, time-bounded validity, lineage, and binding) to uncover structural pipeline weaknesses before higher-level threat modeling takes place. Ultimately, TCTM bridges the gap between theoretical modeling and operational defense by translating telemetry vulnerabilities into tangible engineering artifacts, including SIEM detection rules, SOAR playbook constraints, and practical mobile testing scenarios.
View details
A Framework for Interactive Machine Learning and Enhanced Conversational Systems
Jerry Young
Richard Abisla
Sanjay Batra
Mikki Phan
Nature, Springer-Verlag (2026)
Preview abstract
Conversational systems are increasingly prevalent, yet current versions often fail to support the full range of human speech, including variations in speed, rhythm, syntax, grammar, articulation, and resonance. This reduces their utility for individuals with dysarthria, apraxia, dysphonia, and other language and speech-related disabilities. Building on research that emphasizes the need for specialized datasets and model training tools, our study uses a scaffolded approach to understand the ideal model training and voice recording process. Our findings highlight two distinct user flows for improving model training and provide six guidelines for future conversational system-related co-design frameworks. This study offers important insights on creating more effective conversational systems by emphasizing the need to integrate interactive machine learning into training strategies.
View details
VIP-MINGLE: A Corpus for Videoconference and In-Person Multimodal Interaction in Group Language Engagement
Abhinay K Bodi
Wenxin Deng
Junrui Huang
Venu G Kadamba
Sumanth B H Karanam
Dhiwahar A Kennady
David Poeppel
Dustin Freeman
Interspeech (2026)
Preview abstract
Group conversations are a fundamental yet complex form of social interaction central to human cognition and telecommunication technology. While understanding and facilitating these interactions has been a long-standing goal, findings are often isolated within specific in-person or videoconferencing settings due to a scarcity of datasets that bridge the two. We introduce VIP-MINGLE, a multimodal dataset comprising 59 hours of recordings (32 groups, 105 participants), featuring paired within-subject sessions in both settings. The dataset includes raw audio/video, psychometric data, processed multimodal features (e.g., diarized speech, facial expressions, transcriptions), and time-resolved human annotations. Our analysis reveals significant behavioral distribution shifts across multiple modalities between settings, reinforcing the need for a cross-setting corpus. VIP-MINGLE serves as a critical resource for developing robust models of group conversations across settings.
View details
Extreme Left Shift: Test Points at RTL Generator
Rajesh Gottumukkala
Veerabhadra Rao Vasa
2026
Preview abstract
Modern SoCs face escalating test costs due to high Test Cycles Per Fault (TCPF) caused by architectural bottlenecks like shared buses and deep decode logic. This work introduces an "Extreme Left Shift" strategy by implementing Design-Aware Test Points (DATP) directly at the RTL Generator level.
By natively integrating enhancements such as Address Parallelism and Direct APB Control, this approach bypasses generic structural constraints. Results show a 4X improvement in TCPF and pattern count for CSR-heavy designs. This methodology delivers superior ATPG Quality of Results (QoR) and significant test cost savings compared to traditional downstream EDA gate-level solutions.
View details
