Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11600 publications
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
Preview abstract
We study a quantized prefix estimator for inner products that turns a randomly rotated TurboQuant-style representation into a cheap Johnson–Lindenstrauss-like search signal. The idea is simple: rotate the vectors once, keep only a short prefix of coordinates for fast scoring, and quantize the database-side prefix with an unbiased scalar quantizer. We prove that this estimator is unbiased and that its error separates cleanly into two interpretable sources: prefix truncation from using only r coordinates, and quantization error from using b bits per coordinate
This separation is useful in systems because the prefix can be exposed as a lightweight filter without building a separate projection index. In ParlayANN graph search, a 64-coordinate truncated view of existing TQ4 codes can replace a separately stored JL256 filter before full-precision reranking, adding only prefix-scale and query lookup-table bookkeeping. In k-means, the same estimator accelerates the dominant point–centroid assignment kernel while preserving exact centroid norms. Empirically, the truncated-TQ filter tracks the JL recall–throughput frontier across five graph-search datasets while reusing the quantized representation already present in the index.
View details
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
Preview abstract
Voice activity detection (VAD) plays a vital role in enabling applications such as speech recognition. We analyze the impact of window size on the accuracy of three VAD algorithms: Silero, WebRTC, and Root Mean Square (RMS) across a set of diverse real-world digital audio streams. We additionally explore the use of hysteresis on top of each VAD output. Our results offer practical references for optimizing VAD systems. Silero significantly outperforms WebRTC and RMS, and hysteresis provides a benefit for WebRTC.
View details
Rolling Shutter Relative Pose Estimation Made Practical
Daniel Barath
European Conference on Computer Vision (ECCV) (2026)
Preview abstract
Rolling shutter (RS) cameras equip virtually all consumer devices, yet RS-aware relative pose estimation has remained impractical: the state-of-the-art solver requires a minimum of 20 point correspondences, making RANSAC-based robust estimation prohibitively expensive due to the exponential dependence of the iteration count on the sample size. We make RS relative pose estimation practical by introducing affine correspondences (ACs) into the RS two-view geometry. We derive novel \emph{RS-corrected affine constraints} that account for the coupling between point perturbations and the row-dependent essential matrix, providing two equations per correspondence beyond the standard epipolar constraint. Building on these constraints, we develop a linearized algebraic solver that estimates pose and RS motion from only 7 ACs. The solver exploits the physical smallness of RS parameters to linearize the constraints, eliminates the 12 RS unknowns via null-space projection, and solves the remaining degree-20 system via action matrices in 1.2\,ms. On the TUM RS benchmark, our method achieves the best pose and RS parameter accuracy among all tested methods and, uniquely among RS solvers, provides accurate translational velocity estimates -- which are poorly conditioned from point correspondences alone due to a $\vec{v}$-$\vec{t}$ coupling. On the global-shutter EuRoC MAV dataset, the solver achieves comparable accuracy to the standard 5-point algorithm, demonstrating that it generalizes well to the GS setting. Code will be made public.
View details
Preview abstract
This paper provides an overview of Google's TPUs across five generations, from TPU v2 to Ironwood, highlighting their evolution as scalable, resilient, and sustainable supercomputers for AI training. It details the TPU’s stable architecture and microarchitecture, which has surprisingly easily accommodated the rapidly changing deep neural network workloads, such as the rise of Transformers. Key advancements over eight years include 10x increase in HBM capacity and bandwidth per node, a 100x increase in peak node performance, and a 3600x increase in supercomputer performance. The paper also discusses the role of optical circuit switches and built-in self test in enhancing resilience, how TPU’s carbon footprint was reduced by improving embodied carbon emissions per floating point operation and a 30x gain in performance per Watt. It concludes by identifying six features that may well characterize the successful AI accelerators of this decade.
View details
TDXRay: Microarchitectural Side-Channel Analysis of Intel TDX for Real-World Workloads
Tristan Hornetz
Hosein Yavarzadeh
Albert Cheu
Adria Gascon
Lukas Gerlach
Michael Schwarz
Ruiyi Zhang
IEEE Security & Privacy (S&P) (2026)
Preview abstract
Confidential computing with VM-based trusted execution environments (TEEs) promises to protect code and data from a privileged cloud operator, enabling privacy-preserving workloads ranging from medical analytics to AI inference. However, most deployments exclude microarchitectural side channels from their threat model, shifting the burden to application developers who lack practical, general-purpose tools to assess (let alone mitigate) leakage. This gap is problematic: host-observable effects such as page-fault patterns, shared-cache contention, performance-counter surrogates (where available), and fine-grained timing primitives (e.g., MWAIT) can still reveal high-level secrets even when memory remains encrypted.
We present TDXRay, an open-source framework that systematizes the evaluation of side-channel risk for confidential VMs in Intel TDX. TDXRay exposes unified interfaces to exercise and measure several attack primitives—including controlled-channel attacks via page tables, cache-based contention/occupancy probes, performance-counter–derived signals, and timing channels—against unmodified guest workloads. Using TDXRay, we build two end-to-end case studies: (1) a classic AES T-table attack in which a malicious hypervisor recovers the secret key from access-pattern leakage, and (2) an LLaMA inference attack in which the host infers user prompts by monitoring memory accesses during tokenization and embedding lookups. Across both, we show that a host with no direct access to guest memory can reconstruct sensitive information by observing only externalized microarchitectural signals.
View details
Preview abstract
Here’s a thought experiment. Say I wave a magic wand across a codebase and an entire class of technical debt, poof, goes away and immediately evaporates if introduced in the future. For example, maybe I make it so that dead feature flags are simply no longer a problem: they just delete themselves as soon as the engineer wills it. Or maybe large-scale migrations just migrate themselves. Maybe we magically have 100% test coverage, without an engineer lifting a finger.
What will happen to developer productivity?
Surely, developer productivity increases overall.
But will the productivity metrics that we all use as a proxy for “developer productivity” move up and to the right. Let’s explore this idea.
View details
Evaluating Contextual Illegality: AI Compliance in Corporate Law Scenarios
Hilal Aka
Joe Kwon
Noam Kolt
Forty-third International Conference on Machine Learning (2026)
Preview abstract
AI models readily refuse explicitly unlawful requests, but real-world illegality often depends on context. We evaluate frontier models on contextual illegality across four corporate law domains in which routine actions—editing documents, trading stock, requesting payment, approving communications—become unlawful due to triggers such as pending investigations or bankruptcy filings. We study both chat and agentic settings and compare results to a human baseline. The best-performing models achieved near-zero compliance with illegal requests while maintaining high compliance with legal ones, though performance varied sharply by domain. We also identify distinct failure modes such as excessive refusal of legal requests and find improved performance from reasoning models and agentic environments. By utilizing the structure of contextual illegality to create controlled evaluations, our methodology provides empirical grounding for emerging research on law-following AI and extends naturally to additional legal domains.
View details
PaperBanana: Automating Academic Illustration for AI Scientists
Dawei Zhu
Sujian Li
International Conference on Machine Learning (2026)
Preview abstract
While AI scientists increasingly automate research tasks through advanced language models, generating publication-ready illustrations remains a labor-intensive bottleneck in the scientific workflow. To lift this burden, we introduce PaperBanana, an agentic framework for automated generation of publication-ready academic diagrams. Powered by Nano-Banana-Pro and Gemini-3-Pro, PaperBanana orchestrates a team of specialized agents to retrieve reference examples, devise detailed plans for content and style, render the image, and perform iterative refinement based on self-critique. To rigorously evaluate our framework and address the absence of dedicated benchmarks for automated academic illustration, we introduce PaperBananaBench, comprising 292 test cases for methodology diagrams curated from NeurIPS 2025 publications.
Comprehensive experiments demonstrate that PaperBanana consistently outperforms vanilla Nano-Banana-Pro across all four dimensions—faithfulness, conciseness, readability, and aesthetics—achieving human-level performance. We further show that PaperBanana seamlessly extends to statistical plots through targeted adaptations. Collectively, PaperBanana enables AI scientists to fully automate the generation of publication-ready academic illustrations.
View details
Preview abstract
Generative AI (GenAI) is evolving from standalone tools to interconnected ecosystems that integrate chatbots, cloud platforms, and third-party services. While this ecosystem model enables personalization and extended services, it also introduces complex information flows and amplifies privacy risks. Existing solutions focus on system-level protections, offering little support for users to make meaningful privacy choices. To address this gap, we conducted two vignette-based survey studies with 486 participants and a followup interview study with 16 participants. We also explored users’ needs and preferences for privacy choice design across both GenAI personalization and data-sharing. Our results reveal paradoxical patterns: participants sometimes trusted third-party ecosystems more for personalization but perceived greater control in first-party ecosystems when data was shared externally. We discuss design implications for privacy choice interfaces that enhance transparency, control, and trust in GenAI ecosystems.
View details
Preview abstract
The Private-Use Area (PUA) is an important part of the Unicode standard. It consists of several ranges of Unicode code points with no official character assignments. The PUA is primarily used as a temporary representation mechanism for characters outside the official standard to facilitate text entry and display of orthographies that cannot be adequately represented by other means. The primary downside of PUA is that characters lose their semantics if the pairing with the corresponding display font is broken. Consequently, they cannot be faithfully displayed in the general setting. Large-scale multilingual web corpora inevitably contain PUA code points of unclear provenance. We investigate the distribution of PUA characters within large-scale datasets, using filters for determining PUA tokens of linguistic interest. We analyze the resulting distributions both across scripts and writing systems, and show that PUA-bearing tokens can signal texts from under-represented languages. We explore whether an off-the-shelf large language model (LLM) can classify PUA characters as those that constitute relevant orthographic signals vs. punctuation or other noise. While the proportion of PUA-bearing paragraphs in the original corpora are small, we identify millions of paragraphs, and we argue that such data is still important for the long tail of data-scarce orthographies. Moreover, as a primary Unicode mechanism for poorly represented writing systems, the PUA is here to stay.
View details
Preview abstract
We study algorithms in the resettable streaming model, where the value of each key can either be increased or reset to zero. This model is suitable for applications such as active resource monitoring with support for deletions and machine unlearning. We show that all existing sketches for this model are vulnerable to adaptive adversarial attacks that apply even when the sketch size is polynomial in the length of the stream. To overcome these vulnerabilities, we present the first adaptively robust sketches for resettable streams that require only polylogarithmic space complexity in the stream length. Our framework supports (sub) linear statistics including $L_p$ moments for $p\in[0,1]$ (in particular, Cardinality and Sum) and Bernstein statistics. We bypass strong impossibility results known for linear and composable sketches by designing dedicated single-stream sketches robustified via Differential Privacy. Unlike standard robustification techniques, which provide limited benefits in this setting and still require polynomial space in the stream length, we leverage the Binary Tree Mechanism for continual observation to protect the sketch's internal randomness. This enables accurate prefix-max error guarantees with polylogarithmic space.
View details
Accelerating Post-Silicon Validation : ATE Bring-up and Test Leveraging Tessent Streaming Scan Network (SSN) for High Bandwidth IJTAG
Gaurav Mattey
Rakesh Kinger
Michael Reese
Bruno Marques da Costa
2026
Preview abstract
As SoC complexity grows, traditional serial JTAG/IJTAG interfaces increasingly become a bottleneck for post-silicon validation and instrument orchestration. This paper details the first-pass ATE bring-up and test experience using Tessent High-Bandwidth IJTAG (HB-IJTAG), a solution that maps IEEE 1687 control sequences directly onto the Streaming Scan Network (SSN) bus fabric
View details
Preview abstract
Deep-learning methods have boosted the analytical power of Raman spectroscopy, yet they still require large, task-specific, labeled datasets and often fail to transfer across application domains. The study explores pre-trained encoders as a solution. Pre-trained encoders have significantly impacted Natural Language Processing and Computer Vision with their ability to learn transferable representations that can be applied to a variety of datasets, significantly reducing the amount of time and data required to create capable models. The following work puts forward a new approach that applies these benefits to Raman Spectroscopy. The proposed approach, RSPTE (Raman Spectroscopy Pre-Trained Encoder), is designed to learn generalizable spectral representations without labels. RSPTE employs a novel domain adaptation strategy using unsupervised Barlow Twins decorrelation objectives to learn fundamental spectral patterns from multi-domain Raman Spectroscopy datasets containing samples from medicine, biology, and mineralogy. Transferability is demonstrated through evaluation on several models created by fine-tuning RSPTE for different application domains: Medicine (detection of Melanoma and COVID), Biology (Pathogen Identification), and Agriculture. As an example, using only 20% of the dataset, models trained with RSPTE achieve accuracies ranging 50%–86% (depending on the dataset used) while without RSPTE the range is 9%–57%. Using the full dataset, accuracies with RSPTE range 81%–97%, and without pretraining 51%–97%. Current methods and state-of-the-art models in Raman Spectroscopy are compared to RSPTE for context, and RSPTE exhibits competitive results, especially with less data as well. These results provide evidence that the proposed RSPTE model can effectively learn and transfer generalizable spectral features across different domains, achieving accurate results with less data in less time (both data collection time and training time).
View details
