Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11612 publications
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
Preview abstract
We study a quantized prefix estimator for inner products that turns a randomly rotated TurboQuant-style representation into a cheap Johnson–Lindenstrauss-like search signal. The idea is simple: rotate the vectors once, keep only a short prefix of coordinates for fast scoring, and quantize the database-side prefix with an unbiased scalar quantizer. We prove that this estimator is unbiased and that its error separates cleanly into two interpretable sources: prefix truncation from using only r coordinates, and quantization error from using b bits per coordinate
This separation is useful in systems because the prefix can be exposed as a lightweight filter without building a separate projection index. In ParlayANN graph search, a 64-coordinate truncated view of existing TQ4 codes can replace a separately stored JL256 filter before full-precision reranking, adding only prefix-scale and query lookup-table bookkeeping. In k-means, the same estimator accelerates the dominant point–centroid assignment kernel while preserving exact centroid norms. Empirically, the truncated-TQ filter tracks the JL recall–throughput frontier across five graph-search datasets while reusing the quantized representation already present in the index.
View details
Observational constraints on net radiative forcing confirm aviation contrail warming
Aarón Sonabend
Joe Yue-Hei Ng
arXiv (2026)
Preview abstract
Contrail cirrus represents a critical component of aviation’s non-CO2 climate impact, but its net radiative forcing, the balance between longwave warming and shortwave cooling, remains poorly constrained by direct observations. As a result, current assessments rely almost exclusively on microphysical models such as CoCiP and global climate simulations. Existing empirical estimates are largely restricted to young, linear tracks, because satellite detection masks have a poor recall of contrails once they spread and merge with natural cirrus, leaving a structural gap in our understanding of long-lived, non-linear contrail cirrus. To address this we use a causal framework that isolates the net radiative contrail effect of flight traffic over the Americas. Building on recent progress that quantified the longwave warming contrail effect using advected flight paths as a proxy for contrails, we expand this continuous treatment approach to capture the highly skewed shortwave cooling impact, delivering a 12-hour lifespan net observational radiative forcing. Our analysis reveals a statistically significant net warming energy forcing of 33.7 (95% CI: 20.8, 47.8) GJ/km flown from April 2019 to April 2020, providing a large-scale empirical quantification of long contrail lifespan impact of the same order as, though somewhat larger than, previous bottom-up simulation estimates. This observational benchmark offers an independent line of evidence on the sign and magnitude of the climate impact of contrails.
View details
CSIG: Congestion Signaling for Datacenter Transports
Abhiram Ravi
Brad Karp
MJ Akhbari
Weida Huang
Konstantinos Prasopoulos
2026
Preview abstract
Optimizing burst-heavy datacenter workloads necessitates fine-grained network control and visibility. We introduce CSIG, a protocol that delivers precise, multi-bit bottleneck congestion signals via a fixed-length Ethernet header. The architecture captures 𝜇s-granularity switch metrics, such as available bandwidth, and signals them to end-hosts using in-band, line-rate operations. We propose Fast Ramp-Up, a congestion control primitive that leverages these bottleneck signals to reduce median RPC latency by 20% and unclaimed bandwidth by 60% in production. Beyond transport-level performance, CSIG enables flow-aware observability by embedding 𝜇s-scale metrics into every packet, allowing individual application transfers to pinpoint their bottleneck location, such as the topology tier limiting their performance. CSIG thus transforms network telemetry from post-hoc correlation into a real time, contextaware capability. We demonstrate CSIG’s broad deployability by validating it across five generations of commodity switch hardware (up to 102.4 Tbps), four NIC generations, and five transport stacks. Our design proves that a streamlined Layer 2 approach, focusing exclusively on the principal path bottleneck, provides transport-agnostic gains without requiring forklift hardware upgrades.
View details
Preview abstract
Massive KV caches can cause severe memory-bandwidth bottlenecks during long-context decoding. Sparse attention methods mitigate this via selective loading, but that comes at a cost: rigid heuristics drop necessary context, leading to quality degradation. We introduce \textbf{Elastic Threshold Attention (ETA)}, an end-to-end trainable architecture that achieves hardware-accelerated decoding speed without sacrificing dense model quality. ETA predicts dynamic, contextual thresholds directly from query representations, allowing the model to allocate dense-like context to difficult retrieval or reasoning steps while pruning routine tokens. To learn this policy from scratch without representation collapse, ETA \emph{multiplicatively suppresses} sub-threshold logits toward zero during training rather than deleting them. Training against this smooth uniform attention floor provides a distributed probability reservoir that \textbf{causes localized attention sinks on initial tokens to disappear}. It also enables the model to hard-prune uninformative KV blocks at inference time and absorb incidental tokens co-admitted by coarse GPU block selection. As a result, a 1.45B pretrained ETA model rivals dense attention across language modeling, commonsense reasoning, and long-context needle retrieval at $\approx 85\%$ training sparsity and $\approx 38\%$ active decode density. At inference time, we implement a custom decode kernel in Triton that screens KV blocks in $O(1)$ time using cached geometric-probabilistic bounds, delivering up to $2.5\times$ wall-clock decode speedups over FlashAttention-2 on sequences up to 512K tokens. Finally, we introduce an offline calibration algorithm for domain-specific deployments that freezes per-head constant thresholds to eliminate predictor overhead, cutting attention compute by an additional $27\%$.
View details
Open and Emergent Problems in Agentic Privacy and Security: A Contextual Angle
Sahar Abdelnabi
Borja de Balle Pigem
Sebastian Benthall
Eleanor Birrell
Kamalika Chaudhuri
Madiha Zahrah Choksi
Rachel Cummings
Adam Davies
Dj Dvijotham
Seliem El-Sayed
Ferdinando Fioretto
Matt Franchi
Adria Gascon
Roxana Geambasu
Sahra Ghalebikesabi
Hamed Haddadi
Jamie Hayes
Ashish Hooda
Amir Houmansadr
Somesh Jha
Chloé Kiddon
Tadayoshi Kohno
Abdullatif Köksal
Haoran Li
Tianshi Li
Nathan Malkin
Sarah Meiklejohn
Niloofar Mireshghallah
Sewoong Oh
Katherine Ortiz
Francesco Pinto
Franziska Roesner
Edo Roth
Jacqueline Rowe
Khawaja Shams
Ilia Shumailov
Yan Shvartzshnaider
Dawn Song
Jose Such
Octavian Suciu
Pierre Tholoniat
Trishita Tiwari
Hal Triedman
Ren Yi
Wen Zhang
Xuhui Zhou
Kassem Fawaz
Stefan Mellem
Helen Nissenbaum
Google (2026)
Preview abstract
The vision of an ecosystem of general and highly capable autonomous agents challenges accepted principles of secure and privacy-preserving system engineering. Inspired by the theory of Contextual Integrity, we argue that in order for agents to act appropriately, their behaviors should comply with societal norms and expectations. This manuscript explores contextual approaches to engineering agentic systems that embody an understanding of societal norms and uphold the appropriateness of actions by design. It presents a list of key open research problems in this area to create awareness among the broader research community across academia, government, and industry.
View details
Securing Elliptic Curve Cryptocurrencies against Quantum Vulnerabilities: Resource Estimates and Mitigations
Michael Broughton
Thiago Bergamaschi
Justin Drake
Dan Boneh
PRX Quantum, 7 (2026), pp. 031001
Preview abstract
The expected emergence of cryptographically relevant quantum computers (CRQCs) will represent a singular discontinuity in the history of digital security, with wide ranging impacts. This whitepaper seeks to elucidate specific implications that the capabilities of developing quantum architectures have on blockchain vulnerabilities and potential mitigation strategies. First, we provide new resource estimates for breaking the 256-bit Elliptic Curve Discrete Logarithm Problem over the secp256k1 curve, the core of modern blockchain cryptography. We demonstrate that Shor's algorithm for this problem can execute with either $\leq 1200$ logical qubits and $\leq 90$ million Toffoli gates or $\leq 1450$ logical qubits and $\leq 70$ million Toffoli gates. In the interest of responsible disclosure, we use a zero-knowledge proof to validate these results without disclosing attack vectors. On superconducting architectures with $10^{-3}$ physical error rates and planar connectivity, those circuits can execute in minutes using fewer than half a million physical qubits. We introduce a critical distinction between ``fast-clock'' (such as superconducting and photonic) and ``slow-clock'' (such as neutral atom and ion trap) architectures. Our analysis reveals that the first fast-clock CRQCs would enable ``on-spend'' attacks on public mempool transactions of some cryptocurrencies. We survey major cryptocurrency vulnerabilities through this lens, identifying systemic risks associated with advanced features in some blockchains such as smart contracts, Proof-of-Stake consensus, and Data Availability Sampling mechanism, as well as the enduring concern of ``abandoned'' assets. We argue that technical solutions would benefit from accompanying public policy and discuss various frameworks of ``digital salvage'' to regulate the recovery or destruction of dormant assets while preventing adversarial seizure. We also discuss implications for other digital assets and tokenization as well as challenges and successful examples of the ongoing transition to Post-Quantum Cryptography (PQC). Finally, we urge all vulnerable cryptocurrency communities to join the migration to PQC without delay.
View details
Online Advertising with Spatial Interactions
Yifan Wang
Mingfei Zhao
Proceedings of the ACM Web Conference 2026, Association for Computing Machinery, New York, NY, USA, 213–224
Preview abstract
Online advertising platforms must decide how to allocate multiple ads across limited screen real estate, where each ad's effectiveness depends not only on its own placement but also on nearby ads competing for user attention. Such spatial externalities — arising from proximity, clutter, or crowding — can significantly alter welfare and revenue outcomes, yet existing auction and allocation models typically treat ad slots as independent or ordered along a single dimension. We introduce a new framework for spatial externalities in online advertising, in which the value of an ad depends on both its slot and the configuration of surrounding ads. We model ad slots as points in a metric space, and model an advertiser's value as a function of both their bid and a discount factor determined by the configuration of other displayed ads.
Within this framework, we analyze two natural models. For the Nearest-Neighbor model, where the value suppression depends only on the closest neighboring ad, we present a polynomial-time algorithm that achieves a constant approximation for the general case. We show that the allocation rule is monotone and can be implemented as a truthful mechanism. For a structured setting of 2D Euclidean space, we provide a PTAS. In contrast, for the Product-Distance model, where interference is aggregated multiplicatively across all neighbors, we establish a strong (and nearly-tight) hardness of approximation -- no polynomial-time algorithm can achieve any polynomial-factor approximation unless P=NP, via a reduction from Max-Independent-Set. Our results provide a foundation for reasoning about spatial externalities in ad allocation and for designing efficient, truthful mechanisms under such interactions.
View details
VidMap: Exploiting Temporal Structure for Video-Based Structure-from-Motion
Zador Pataki
Paul-Edouard Sarlin
Marc Pollefeys
ECCV (2026)
Preview abstract
Accurately recovering the camera's calibration and metric poses for any unconstrained video would unlock large-scale training data for navigation and scene understanding. The dominant approaches to this problem are severely limited: Simultaneous Localization and Mapping (SLAM) is sensitive to initialization and transient failures due to its causal, incremental nature; it is often over-optimized for real-time operation and generally requires known camera calibration; while Structure-from-Motion (SfM) typically forgoes any image ordering, enabling optimal initialization and global optimization, but lacks robustness to visual symmetries and extreme motions. To bridge this gap, we introduce a system that combines the strong sequential constraints of SLAM with the flexibility and global optimization of offline SfM, enabling the metric reconstruction of arbitrary, long, uncalibrated videos. This system leverages recent advances in wide-baseline dense image matching, treats temporal ordering as a first-class citizen for reliable loop closure, and augments global optimization with metric monocular depth priors. As a result, thorough evaluations on diverse, challenging datasets that exhibit extreme motion and visual symmetries reveal that our approach is significantly more robust and accurate than both state-of-the-art SLAM and SfM, classical or learned, with given or unknown camera calibration. The code is publicly available at https://github.com/cvg/vidmap.
View details
Editing Everything Everywhere All at Once
Fabio Quattrini
Carmine Zaccagnino
Enis Simsar
Marta Tintore Gazulla
Rita Cucchiara
Silvia Cascianelli
2026
Preview abstract
Editing multiple elements of an image in a single forward pass has recently emerged as a practical alternative to multi-turn image manipulation, offering improved efficiency. However, when several instructions target different regions, semantic interference often leads to attribute leakage and poor edit disentanglement, especially as the number of edits increases. In this work, we propose MICE (Multi-Instance Concurrent Editing), a training-free strategy for scalable multi-instance image editing with Multimodal Diffusion Transformers. MICE modifies the additive bias of joint attention to regulate interactions between instance-specific text, latent, and context tokens identified via user-provided segmentation masks. Specifically, MICE allows intra-instance attention, penalizes interactions between neighboring region tokens, and suppresses unrelated cross-instance attention. As a result, our method enforces attribute binding while preserving global visual consistency. We evaluate MICE on LoMOE-Bench and introduce MICE-Bench, a more challenging benchmark with an average of 8.5 concurrent edits per image. The experiments demonstrate that our approach outperforms strong baselines and recent competitors in terms of the number of attempted edits and faithfulness to the textual editing instruction.
View details
Preview abstract
When managing complex, unpredictable (non-deterministic) AI agents using simple, fixed control systems (like finite state machines), operational failures and accountability issues often arise. This document introduces a probabilistic governance and telemetry framework to resolve these problems. Instead of following a rigid sequence of steps, this framework defines a multi-dimensional operational boundary, a 'behavioral volume', and assigns the agent a goal. This allows the agent to use its own reasoning to achieve the goal while remaining within the defined boundaries. A separate telemetry layer monitors the agent's actions by calculating metrics, such as alignment scores and drift velocity, to measure how much the agent deviates from its intended behavior. This system provides a method for guiding, monitoring, and securing autonomous agents, effectively managing the performance and security of an unpredictable AI workforce in complex environments.
View details
Fine-Grained Table Retrieval for Open-Domain Tabular Question Answering
Xingyu Ji
Wojciech Kosiuk
Madelon Hulsebos
Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026), Association for Computational Linguistics
Preview abstract
This work introduces a fine-grained table retrieval framework for grounding large
language models in heterogeneous, open-domain relational data. Instead of encoding a query as a single vector, the approach decomposes natural
language queries into semantic components and embeds each independently, enabling
more precise matching of compositional query intent. These representations are used in a staged retrieval pipeline with component-level
search, connectivity-aware grouping, and reranking. Experiments on three TARGET benchmark corpora show consistent improvements in capped recall@k and stronger alignment between query intent and tabular structure over dense retrieval baselines, particularly for longer and more complex queries and when using lightweight embedding models.
View details
Securing the Agentic Enterprise
Amazon Kindle (2026)
Preview abstract
Securing the Agentic Enterprise: Threat Modeling, Anomaly Detection, and Governing Autonomous Multi-Agent Systems addresses the critical security and governance gaps emerging as enterprises transition from human-supervised copilots to autonomous agentic workflows. As software processes gain the ability to reason, decompose natural language objectives, and execute multi-step tool calls at machine speed, traditional syntactic security boundaries (like firewalls and static analysis) become obsolete.
This book provides security architects, CISOs, and platform engineers with a practical, architecture-level blueprint for securing this new paradigm. It explores novel attack vectors such as indirect prompt injections and consumption-based economic threats and provides frameworks for robust mitigation. Key topics include modernizing agentic identity, implementing semantic firewalls, transition-state anomaly detection, and applying zero-trust principles to autonomous execution contexts. Bridging the gap between high-level ethical guidelines and isolated model safety, this guide prepares practitioners to confidently deploy and govern enterprise-grade autonomous systems.
View details
Preview abstract
We introduce a new Bayesian perspective on the concept of data reconstruction, and leverage this viewpoint to propose a new security definition that, in certain settings, provably prevents reconstruction attacks. We use our paradigm to shed new light on one of the most notorious attacks in the privacy and memorization literature - fingerprinting code attacks (FPC). We argue that these attacks are really a form of membership inference attacks, rather than reconstruction attacks. Furthermore, we show that if the goal is solely to prevent reconstruction (but not membership inference), then in some cases the impossibility results derived from FPC no longer apply.
View details
