Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11631 publications
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
Preview abstract
The rapid expansion of the Internet of Things (IoT) and smart home ecosystems has led to a fragmented landscape of user data management across consumer electronics (CE) such as Smart TVs, gaming consoles, and set-top boxes. Current onboarding processes on these devices are characterized by high friction due to manual data entry and opaque data-sharing practices. This paper introduces the User Data Sharing System (UDSS), a platform-agnostic framework designed to facilitate secure, privacy-first PII (Personally Identifiable Information) exchange between device platforms and third-party applications. Our system implements a Contextual Scope Enforcement (CSE) mechanism that programmatically restricts data exposure based on user intent—specifically distinguishing between Sign-In and Sign-Up workflows. Unlike cloud-anchored identity standards such as FIDO2/WebAuthn, UDSS is designed for shared, device-centric CE environments where persistent user-to-device bind-ing cannot be assumed. We further propose a tiered access model that balances developer needs with regulatory compliance (GDPR/CCPA). A proof-of-concept implementation on a reference ARMv8 Linux-based middleware demonstrates that UDSS reduces user onboarding latency by 65% and measurably reduces PII over-exposure risk through protocol-enforced data minimization. This framework provides a standardized approach to identity management in the heterogeneous CE market.
View details
Nudging Developers Toward Privacy: Evaluating the Impact of Personalized App Review Reports
Omer Akgul
Michelle L. Mazurek
USENIX Symposium on Usable Privacy and Security (SOUPS) (2026)
Preview abstract
Mobile application developers often struggle to create accurate privacy notices or implement robust privacy practices due to limited expertise or resources. While users share unsolicited privacy feedback in app reviews, and prior research has characterized this privacy feedback, uncovering developer reactions to this feedback remains unexplored. This study explores whether personalized privacy review reports---summarizing real user feedback for a developer's own app---can effectively nudge them toward planning privacy improvements. We surveyed 42 app developers, presenting them with reports containing privacy themes, temporal trends, peer benchmarks, and emotion distributions derived from their apps' reviews. Our findings indicate that these privacy report interventions proved highly effective, with 76% (32 of 42) of participants finding at least one section of the report useful. Furthermore, exposure to the report increased the participants' intent to pursue privacy-relevant actions -- such as reorganizing the UI, enhancing privacy communications, or adding/removing features -- with 69% (29 of 42) of participants indicating an increased intent to do so. Almost all developers expressed a desire to receive such privacy reports periodically or on demand. These results indicate that making this style of report broadly available across the industry could foster a more privacy-conscious mobile ecosystem.
View details
MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR
Sieun Kim
Qianhui Zheng
Ruoyu Xu
Ravi Tejasvi
Anuva Kulkarni
Junyi Zhu
2026
Preview abstract
In Extended Reality (XR), complex acoustic environments often overwhelm users, compromising both scene awareness and social engagement due to entangled sound sources. We introduce MoXaRt, a real-time XR system that uses audio-visual cues to separate these sources and enable fine-grained sound interaction. MoXaRt's core is a cascaded architecture that performs coarse, audio-only separation in parallel with visual detection of sources (e.g. faces, instruments). These visual anchors then guide refinement networks to isolate individual sources, separating complex mixes of up to five concurrent sources (e.g. two voices + three instruments) with ca. 2 second processing latency. We validate MoXaRt through a technical evaluation on a new, complex dataset we collected, and a 22-participant user study. Our results demonstrate that MoXaRt significantly improves communication clarity—boosting listening comprehension in noisy conditions by 33.2% (p=0.0058)—and significantly reduces cognitive load (M=7.50 vs. M=3.36, p<0.001), paving the way for more perceptive and socially adept XR experiences.
View details
Preview abstract
Superconducting qubits are a leading platform for realizing fault-tolerant quantum computers. Current generations demonstrate fast, high fidelity quantum gates and readout on the order of hundreds of nanoseconds, while maintaining coherence times exceeding one hundred microseconds. Achieving this state-of-the-art performance requires a tight co-design, balancing fundamental physics, microwave engineering, and semiconductor fabrication. Readout designs, in particular, benefit from this multidisciplinary approach.
In this talk, we discuss the current challenges for readout in superconducting quantum processors from the perspective of Google Quantum AI. We examine the intersection of device physics and microwave engineering constraints, illustrating how optimizing both is essential for scaling next-generation quantum systems.
View details
Preview abstract
While non-verbal behaviors and expressive movements are essential for natural human-robot interaction, existing methods often overlook a crucial element: the human’s internal cognitive state. Consequently, proactive multi-agent systems frequently interrupt humans at inopportune moments, leading to cognitive overload and decreased task performance. This paper introduces a framework for generating “cognitively aligned” multi-agent interactions, enhancing the ability of robotic systems to contextually defer communications during moments of high human mental workload. We present the design and implementation of a closed-loop architecture that explores the interplay between autonomous task execution and real-time neurophysiological focus. Utilizing a consumer-grade Brain-Computer Interface (BCI), our approach continuously monitors Electroencephalography (EEG) spectral band powers while a human performs a cognitive-load-inducing task. We propose a workload-driven pipeline where an HTTP-based signaling mechanism places a primary agent’s sensory inputs and audio outputs into a holding state upon detecting high cognitive load. This allows secondary agents to seamlessly process complex, delegated tasks in the background. Once the human’s cognitive state returns to a baseline, the primary agent releases the queued agent message. Our preliminary results demonstrate the feasibility of leveraging real-time signal processing, Large Language Models (LLMs), and physical robotic embodiments to create interrupt-aware, non-intrusive multi-agent systems.
View details
What’s on My Network? Using Large Language Models to Identify Real-World IoT Devices at Scale
Rameen Mahmood
Danny Yuxing Huang
Proceedings of ACM International Conference on Emerging Networking Experiments and Technologies (CoNEXT), Association for Computing Machinery (2026)
Preview abstract
Large-scale cloud-native systems generate continuous streams of operational alerts across distributed microservice architectures. On-call engineers must manually triage these alerts by correlating signals from heterogeneous observability tools, a process that is time-consuming, cognitively demanding, and prone to error. Despite advances in monitoring and anomaly detection, incident triage remains largely manual. This paper presents a declarative, large language model (LLM)–driven multi-agent approach to automating incident triage and Service Level Objective (SLO) monitoring. The proposed design constrains agent behavior using domain-expertauthored investigation workflows, enabling deterministic execution and reproducibility while preserving operational safety. The framework integrates a unified tool execution layer for interacting with diverse observability systems and an enhanced retrieval-augmented generation (RAG) pipeline optimized for operational knowledge retrieval. The approach has been evaluated in a production cloud environment spanning multiple microservices and geographic regions. Results show reductions in high-severity incident triage time from approximately 30 minutes to under 5 minutes, alert acknowledgement latency from minutes to seconds, and service onboarding effort from weeks to days. These findings suggest that constrained multi-agent systems can substantially reduce on-call cognitive load while maintaining reliability and human oversight.
View details
Preview abstract
We study the stochastic linear bandits with parameter noise model, in which the reward of action $a$ is $a^\top \theta$ where $\theta$ is sampled i.i.d.
We show a regret upper bound of $\tO \brk{\sqrt{d T \logKdelta \sigmax}}$ for a horizon $T$, general action set of size $K$ of dimension $d$, and where $\sigmax$ is the maximal variance of the reward for any action.
We further provide a lower bound of $\tOmega \brk{d \sqrt{T \sigmax}}$ which is tight (up to logarithmic factors) whenever $\log \brk{K} \approx d$.
For more specific action sets, $\ell_p$ unit balls with $p \leq 2$ and dual norm $q$, we show that the minimax regret is $\tTheta \brk{\sqrt{dT \vsigq}}$, where $\vsigq$ is a variance-dependent quantity that is always at most $4$.
This is in contrast to the minimax regret attainable for such sets in the classic additive noise model, where the regret is of order $d \sqrt{T}$.
Surprisingly, we show that this optimal (up to logarithmic factors) regret bound is attainable using a very simple explore-exploit algorithm.
View details
Preview abstract
Despite significant strides in factual reliability, errors -- often termed hallucinations -- remain a major concern for generative AI, especially as LLMs are increasingly expected to be helpful in more complex or nuanced setups. Yet even in the simplest setting -- factoid question-answering with clear ground truth-frontier models without external tools continue to hallucinate. We argue that most factuality gains in this domain have come from expanding the model's knowledge boundary (encoding more facts) rather than improving awareness of that boundary (distinguishing known from unknown). We conjecture that the latter is inherently difficult: models may lack the discriminative power to perfectly separate truths from errors, creating an unavoidable tradeoff between eliminating hallucinations and preserving utility.
This tradeoff dissolves under a different framing. If we understand hallucinations as confident errors -- incorrect information delivered without appropriate qualification -- a third path emerges beyond the answer-or-abstain dichotomy: expressing uncertainty. We propose faithful uncertainty: aligning linguistic uncertainty with intrinsic uncertainty. This is one facet of metacognition -- the ability to be aware of one's own uncertainty and to act on it. For direct interaction, acting on uncertainty means communicating it honestly; for agentic systems, it becomes the control layer governing when to search and what to trust. Metacognition is thus essential for LLMs to be both trustworthy and capable; we conclude by highlighting open problems for progress towards this objective.
View details
Preview abstract
In "Elephants, Goldfish and the New Golden Age of Software Engineering," the author discusses how AI is changing knowledge work, especially software development. Written from the perspective of April 2026, the article points out that while AI speeds up coding, it can also quickly generate a lot of mistakes and messy code if it isn't carefully managed by human oversight and clear processes.
The paper outlines a practical approach to working with AI, broken down into three main sections:
Using AI as a Tool, Not a Toy:
The author notes that people often get poor results by asking AI to do everything in a single prompt. Instead, users should have back-and-forth conversations with AI to question assumptions, set clear grading rules, and guide the research. The main point is that humans must still provide the final judgment; AI is simply a way to speed up and record that thinking.
The Elephant-Goldfish Model:
As AI creates more code than humans can easily read, written design documents become more important than the code itself. To keep AI on track, the author suggests a two-part method:
* The Elephant: A long chat session where the human and AI discuss ideas and write a detailed design document *before* any code is written. This session holds all of the project's background information and decisions.
* The Goldfish: A brand-new AI chat session with no memory. The human asks this "goldfish" to read the design document. If the goldfish cannot understand the plan based only on that document, the document needs more details.
* Only after the design document is clear enough for the goldfish to understand does the human ask the AI to write the code based on those strict instructions.
* Managing AI and the Future of Work:
The author expects that regular employees will soon act like managers, overseeing multiple AI helpers. Because of this, workers need to learn basic management skills, like how to delegate tasks and set clear boundaries. Also, since AI will handle routine chores, humans will need to practice focusing for longer periods to do deeper, harder thinking. Ultimately, a worker's value will come from their planning and decision-making skills, rather than their ability to type code.
View details
Preview abstract
We study the design of the optimal mechanism in a setting with one buyer and one item in which the seller receives a binary signal correlated with the buyer's type after the allocation decision and can condition the payment on the signal realization. The optimal mechanism depends on the correlation between the signal and the buyer's type through the conditional probability of the signal given the type, which we call the \emph{buyer's belief function}. Under the standard monotone likelihood property (MLRP), we establish that the curvature of the buyer's belief function plays a central role: full surplus extraction is achievable if and only if the belief function is convex in the buyer's type. In contrast, when beliefs are concave, it is optimal to fully allocate to the buyer if and only if her type is outside an intermediary range of types (corresponding to types with a higher correlation with the signal). We further establish that, in the case of concave beliefs,
the optimal mechanism can be implemented as a \emph{state-contingent posted payment} offered on a take-it-or-leave-it basis. We further provide a structural characterization of the optimal deterministic mechanism under general belief functions: the state space partitions into several regions, depending on the curvature of the belief function, and in each region, we either have full surplus extraction or a state-contingent posted payment. Finally, we show how our results can be extended to a setting with multiple buyers.
View details
Accelerating Post-Silicon Validation : ATE Bring-up and Test Leveraging Tessent Streaming Scan Network (SSN) for High Bandwidth IJTAG
Gaurav Mattey
Rakesh Kinger
Michael Reese
Bruno Marques da Costa
2026
Preview abstract
As SoC complexity grows, traditional serial JTAG/IJTAG interfaces increasingly become a bottleneck for post-silicon validation and instrument orchestration. This paper details the first-pass ATE bring-up and test experience using Tessent High-Bandwidth IJTAG (HB-IJTAG), a solution that maps IEEE 1687 control sequences directly onto the Streaming Scan Network (SSN) bus fabric
View details
Preview abstract
Audio Description ( AD) provides essential access to visual media for blind and low vision ( BLV) audiences. Yet current AD production tools remain largely inaccessible to BLV video creators, who possess valuable expertise but face barriers due to visually- driven interfaces. We present ADCanvas, a multimodal authoring system that supports non- visual control
over audio description ( AD) creation. ADCanvas combines conversational interaction with keyboard- based playback control and a plain- text, screen reader–
accessible editor to support end- to- end AD authoring and visual question answering ( VQA). Combining screen- reader- friendly controls with a multimodal
LLM agent, ADCanvas supports live VQA, script generation, and AD modification. Through a user study with 12 BLV video creators, we find that users adopt
the conversational agent as an informational aide and drafting assistant, while maintaining agency through verification and editing. For example, participants
saw themselves as curators who received information from the model and filtered it down for their audience. Our findings offer design implications for
accessible media tools, including precise editing controls, accessibility support for creative ideation, and configurable rules for human- AI collaboration.
View details
