Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11573 publications
A 3D Scene Graphs Survey: Open Challenges and Future Directions
Dennis Rotondi
Francesco Argenziano
Sebastian Koch
Nathan Hughes
Martin Büchner
Johanna Wald
Lukas Schmid
Daniele Nardi
Abhinav Valada
Liam Paul
Luca Carlone
Kai Arras
Annual Review of Control, Robotics, and Autonomous Systems (ARCRAS), 10 (2027) (to appear)
Preview abstract
3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI by combining geometric grounding with semantic and relational abstractions of the environment. Their expressiveness has made them relevant to a broad range of problems in robotics and computer vision, including mapping, task and motion planning, scene understanding, and many others. However, the field remains fragmented: different communities adopt distinct formulations, construction pipelines, and evaluation protocols, making it difficult to compare methods, identify common assumptions, and assess remaining challenges for robust real- world deployment. This survey provides a unified and critical review of 3DSGs, with particular emphasis on open challenges and future directions. We first formalize 3DSGs under a common definition and analyze the principal modeling choices that characterize existing formulations, including node and edge attributes, hierarchical structure, dynamic scene representations, and affordance-aware extensions. We then review how 3DSGs are constructed from raw sensory observations, covering both learning-oriented and construction-oriented systems. Finally, we examine downstream applications and evaluation strategies, from intrinsic graph quality to task-level performance. To support the community, we also provide a dedicated website that organizes and extends the surveyed works.
View details
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
Correct-by-Construction Framework for Robustness Against Divergent Voltage Drops and High Clock Divergence
Prateek Pendyala
Mathew Kaipanatu
2026
Preview abstract
Limitations in Sign-off Methodology:
Traditional STA corner selection 10% lower STA corner from PMIC voltage is selected- design is constantly optimised for 10% lower voltage, thereby failing to build margin against differential drop.
IR aware STA does not account timing path’s geometric imbalances (logic depths), net dominated interconnect skews (net delays & metal layer variation) - all dominant in advanced process nodes. Furthermore, this is workload dependent: fixing IR STA violations does not build margins on unseen vectors.
Frequent Silicon issues due to these gaps:
Low voltage mode scan shift Vmin jumps need to meet slack on paths which become exponentially sensitive to voltage gradients. Even small differential IR drops (capture & launch traversing through contrasting IR hotspot & cool regions) cause catastrophic slack loss
High divergence paths with structural imbalances-where clock paths are net-dominated & are operated at high speeds often fail to meet hold timing, despite good pre-silicon margins due to high interlayer metal-sheet & via resistances in lower process nodes.
Additionally, there is considerable PPA impact -higher dynamic & leakage power in clock & data paths respectively in divergent paths. Our proposed solution aims to address above gaps.
View details
Securing Multi-Agent Systems: An Empirical Analysis of Security Prompt Hardening and Residual Risks
Mithil Roshan
Shashank Nagesh
(2026)
Preview abstract
The rapid adoption of agentic systems powered by
large language models (LLMs) introduces significant
security challenges distinct from plain conversational
models, particularly concerning prompt injection and
tool misuse due to their dynamic personas and real-
world tool interactions. This paper investigates the
effectiveness of hardened security prompting in a
task-oriented multi-agent framework, using a coding
assistant as a representative case study. We com-
pare a baseline ”unhardened” agent against a ”hard-
ened” version equipped with explicit security guide-
lines applied across all sub-agents. Our evaluation
across 150+ single-turn and 32 multi-turn attack sce-
narios demonstrates that prompt hardening dramat-
ically improves resilience. With a simple, approxi-
mately 500-token security hardener, single-turn fail-
ure rates dropped from 19.48% to 2.60%, while multi-
turn failure rates decreased from 75.00% to 46.88%.
Furthermore, we show that successfully bypassing the
hardened agent requires significantly more adversar-
ial effort and a greater number of chat turns. How-
ever, the analysis also reveals a critical shift in vul-
nerability taxonomy: as direct attacks fail, adver-
saries exploit the agent’s core functionality via ”Func-
tional Wrappers” (Intent Obfuscation), highlighting
a residual risk that necessitates a shift in the defen-
sive paradigm from static filters to dynamic runtime
state and intent analysis.
View details
POLCA: Stochastic Generative Optimization with LLM
Xuanfei Ren
Allen Nie
Tengyang Xie
Ching-An Cheng
2026
Preview abstract
Optimizing complex systems, ranging from LLM prompts to multi-turn agents, traditionally requires labor-intensive manual iteration. We formalize this challenge as a stochastic generative optimization problem where a generative language model acts as the optimizer, guided by numerical rewards and text feedback to discover the best system. We introduce Prioritized Optimization with Local Contextual Aggregation (POLCA), a scalable framework designed to handle stochasticity in optimization -- such as noisy feedback, sampling minibatches, and stochastic system behaviors -- while effectively managing the unconstrained expansion of solution space. POLCA maintains a priority queue to manage the exploration-exploitation tradeoff, systematically tracking candidate solutions and their evaluation histories. To enhance efficiency, we integrate an ε-Net mechanism to maintain parameter diversity and an LLM Summarizer to perform meta-learning across historical trials. We theoretically prove that POLCA converges to near-optimal candidate solutions under stochasticity. We evaluate our framework on diverse benchmarks, including τ-bench, HotpotQA (agent optimization), VeriBench (code translation) and KernelBench (CUDA kernel generation). Experimental results demonstrate that POLCA achieves robust, sample and time-efficient performance, consistently outperforming state-of-the-art algorithms in both deterministic and stochastic problems. The codebase for this work is publicly available at this https URL.
View details
System for a Secure, Outcome-Based Synthetic Labor Market Using Trusted Execution Environments
Patent (2026)
Preview abstract
Some artificial intelligence provisioning models that function as tools for human users or rely on labor arbitrage can present challenges for organizations, such as managing personnel rather than task outcomes and introducing data security risks. An architecture is described for an outcome-based synthetic labor market in which autonomous computational agents can be compensated based on verified task completion. The framework can leverage trusted execution environments to create secure hardware enclaves for processing sensitive data, which can render the data cryptographically inaccessible to a host system or agent provider. This approach can facilitate a secure, transactional market for autonomous professional execution, which may enable a shift from managing labor resources to procuring verified outcomes from a pool of specialized agents.
View details
Accurate ground state energy estimation with noise and imperfect state preparation
Tom O'Brien
Alicja Dutkiewicz
Stefano Polla
ArXiv (2026)
Preview abstract
We introduce a classical estimator for the post-processing of quantum phase estimation (QPE) data generated either by quantum-Fourier-transform-based or quantum-signal-processing-based methods. These methods sample from a distribution supported on $[0, 2\pi]$, depending on the location of the target phase within this window. We focus on the case where the target phase lies within a smaller promise region, where no other phases are present, which is typical of e.g. ground state energy estimation of gapped quantum systems. Our estimation technique is based on filtering the signal within the promise region and recovering the phase through a moment-projection estimator. We show that our methods are robust in the presence of both additional phases outside the promise region and global depolarizing noise. We observe in the noiseless case our estimator achieves an exponential improvement over naive mean estimation in certain parameter regimes. In the presence of global depolarizing noise, no robust state of the art is known, and our estimator achieves a bias exponentially small in the circuit depth $T$ at fixed circuit fidelity $F$, and a variance proportional to $T^{-2}$, improving by a factor of $T^4$ over the naive approach. To mitigate realistic circuit-level noise, we combine our method with the explicit unbiasing scheme described in Ref.~\cite{dutkiewiczError2025}. As an illustrative example, we implement these estimators on a small-scale simulation of the Ising model. We observe that the moment-projection estimator retains some residual bias removable by the explicit unbiasing scheme, but this was significantly smaller than the variance in the system for the parameter range tested. The robustness of the moment-projection estimator in the presence of both multiple eigenvalues and noise makes phase estimation with limited depth practical for early fault tolerant quantum experiments.
View details
XMob: Accelerated Differentiable Mesoscopic Traffic Simulation in JAX
Sam Griesemer
John Burge
Shantanu Shahane
Yechen Li
Naples, Italy (2026)
Preview abstract
This paper introduces XMob, a novel differentiable traffic simulation framework built in JAX to advance traditional models like SUMO’s mesoscopic simulator. By leveraging JAX’s capabilities for vectorized, hardware-accelerated computation (GPU/TPU), XMob achieves orders-of-magnitude speedups, enabling large-scale urban network simulations and extensive counterfactual analyses. A key innovation is XMob’s inherent differentiability, facilitating direct integration with gradient-based optimization for tasks such as demand calibration and network parameter estimation, significantly outperforming black-box approaches. Furthermore, XMob can be used in Physics-Informed Machine Learning (PIML) pipelines to enhance data-driven augmentation, embedding domain principles like flow conservation and shockwave theory. This ensures physically plausible and robust predictions, even for unobserved scenarios such as lane modifications. The hybrid architecture, combining a deterministic JAX core with incremental machine learning, offers a scalable and efficient solution for modern traffic simulation and optimization challenges.
View details
Learning Conditional Averages
Marco Bressan
Nataly Brukhim
Nicolo Cesa-Bianchi
Emmanuel Esposito
Shay Moran
Maximilian Thiessen
COLT (2026)
Preview abstract
We introduce the problem of learning \emph{conditional averages} in the PAC framework.
The learner receives a sample labeled by an unknown target concept from a known concept class, as in standard PAC learning. However, instead of learning the target concept itself, the goal is to predict, for each instance, the average label over its \emph{neighborhood}---an arbitrary subset of points that contains the instance.
In the degenerate case where all neighborhoods are singletons, the problem reduces exactly to classic PAC learning. More generally, it extends PAC learning to a setting that captures learning tasks arising in several domains, including explainability, fairness, and recommendation systems. %including explainability, fairness, and recommendation systems.
Our main contribution is a complete characterization of when conditional averages are learnable, together with sample complexity bounds that are tight up to logarithmic factors. The characterization hinges on the joint finiteness of two novel combinatorial parameters, which depend on both the concept class and the neighborhood system, and are closely related to the independence number of the associated neighborhood graph.
View details
Looking to the brain to improve energy efficiency of AI
Taro Toyoizumi
Hakwan Lau
Michał Klincewicz
Seng Bum Michael Yoo
Megan Peters
taylor.w.webb@gmail.com
Current Biology (2026)
Preview abstract
Modern artificial intelligence (AI) systems have achieved remarkable capabilities, but at an extraordinary energy cost. Training and running large-scale models can consume vast resources, posing environmental, economic, and social challenges. In contrast, biological brains perform lifelong learning, adaptive control, and flexible reasoning using orders of magnitude less energy for learning and adaptation over a lifetime. What accounts for this difference -- and how can it guide future AI development? In this article, we identify key biological principles that support energy-efficient capacities in biological brains, and consider how they might inform the design of more sustainable artificial systems. We organize our analysis around three domains: architectural constraints, signaling strategies, and learning algorithms. In each domain, we discuss concrete observations from biology -- from cell to circuit to cognitive level -- and describe how current and emerging AI systems mirror or diverge from these motifs. One striking feature of biological energy optimization is often overlooked: that brains are remarkably stable in their energy usage across heterogeneous modes, suggesting they may minimize energy needs during active environmental processing through maximizing the utility of “rest-like” background processes. Overall, rather than advocating for biomimicry for its own sake, we argue for biologically informed engineering. Understanding how natural systems minimize energetic cost while maximizing flexibility may help us build AI that is not only powerful, but also efficient, equitable, and environmentally responsible.
View details
Preview abstract
Statistical evaluation aims to estimate the generalization performance of a model using held-out i.i.d. test data sampled from the ground-truth distribution. In supervised learning settings such as classification, performance metrics such as error rate are well-defined, and test error reliably approximates population error given sufficiently large datasets. In contrast, evaluation is more challenging for generative models due to their open-ended nature: it is unclear which metrics are appropriate and whether such metrics can be reliably evaluated from finite samples. In this work, we introduce a theoretical framework for evaluating language models and establish evaluability results for commonly used metrics. We study two categories of metrics: test-based metrics, including integral probability metrics (IPMs), and similarity-based metrics, including Rényi and KL divergences. We show that IPMs with respect to any bounded test class can be evaluated from finite samples up to multiplicative and additive approximation errors. Moreover, when the test class has finite fat-shattering dimension, IPMs can be evaluated with arbitrary precision. In contrast, similarity-based metrics, including Rényi and KL divergences, are not evaluable from finite samples, as their values can be critically determined by rare events. We also analyze the potential and limitations of perplexity as an evaluation method.
View details
PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
Pengfei He
Vishesh Sharma
Ash Fox
George Lee
Jiliang Tang
2026
Preview abstract
Large Language Models (LLMs) are rapidly evolving into agentic systems that interact with external tools and dynamic environments, but this also introduces severe security risks. In particular, indirect prompt injection attacks can compromise agents through malicious instructions hidden in external sources such as web pages, emails, and retrieved documents. Existing defenses are largely reactive, while current automated red-teaming methods mainly optimize attack success rather than systematically uncovering hidden vulnerabilities within the agent pipeline. In this work, we propose PI-Hunter, an automated agentic red-teaming framework that shifts the focus from attack optimization to vulnerability exposure. By combining static attack-surface analysis, source-aware seeding, trajectory evaluation, and feedback-guided exploration, PI-Hunter proactively discovers vulnerable ingestion paths and localizes how malicious instructions propagate through agent reasoning. Extensive experiments across multiple benchmarks, agent architectures, attacks, and defenses show that \method~substantially improves vulnerability exposure and attack-surface coverage compared with existing automated red-teaming baselines, while remaining effective even under strong prompt injection defenses.
View details
Mining Attribute Subspaces for Efficient Fine-tuning of 3D Foundation Models
Yu Jiang
Hanwen Jiang
Vincent Chu
Brandon Y. Feng
Zhangyang Wang
Qixing Huang
IEEE/CVF Conference on Computer Vision and Pattern Recognition (2026)
Preview abstract
With the emergence of 3D foundation models, there is growing interest in fine-tuning them for downstream tasks, where LoRA is the dominant fine-tuning paradigm. As 3D datasets exhibit distinct variations in texture, geometry, camera motion, and lighting, there are interesting fundamental questions: 1) Are there LoRA subspaces associated with each type of variation? 2) Are these subspaces disentangled (i.e., orthogonal to each other)? 3) How do we compute them effectively? This paper provides answers to all these questions. We introduce a robust approach that generates synthetic datasets with controlled variations, fine-tunes a LoRA adapter on each dataset, and extracts a LoRA sub-space associated with each type of variation. We show that these subspaces are approximately disentangled. Integrating them leads to a reduced LoRA subspace that enables efficient LoRA fine-tuning with improved prediction accuracy for downstream tasks. In particular, we show that such a reduced LoRA subspace, despite being derived entirely from synthetic data, generalizes to real datasets. An ablation study validates the effectiveness of the choices in our approach.
View details
Preview abstract
We study cooperative multi-agent reinforcement learning in the setting of reward-free exploration, where multiple agents jointly explore an unknown MDP in order to learn its dynamics (without observing rewards). We focus on a tabular finite-horizon MDP and adopt a phased learning framework. In each learning phase, multiple agents independently interact with the environment. More specifically, in each learning phase, each agent is assigned a policy, executes it, and observes the resulting trajectory. Our primary goal is to characterize the tradeoff between the number of learning phases and the number of agents, especially when the number of learning phases is small.
Our results identify a regime change governed by the horizon $H$. When the number of learning phases equals $H$, we present a computationally efficient algorithm that uses only $\tilde{O}(S^6 H^6 A / \epsilon^2)$ agents to obtain an $\epsilon$ approximation of the dynamics (i.e., yields an $\epsilon$-optimal policy for any reward function). We complement our algorithm with a lower bound showing that any algorithm restricted to $\rho < H$ phases requires at least $A^{H/\rho}$ agents to achieve constant accuracy.
Thus, we show that having $\Theta(H)$ learning phases is both necessary and sufficient when restricting the number of agents to be polynomial.
View details
Preview abstract
Forensic verification often uses a binary “real vs. fake” label that groups fully synthetic, tampered, and AI-retouched images despite their different consequences. We study these modifications through two complementary channels: a camera channel sensitive to capture and processing traces, and a semantic channel capturing scene content. The channels provide continuous evidence rather than deterministic signatures of manipulation history. We instantiate this perspective in 2CAP (2-Channel Authenticity Protocol), pairing a contrastively trained camera encoder with a frozen semantic encoder for (i) reference-free four-class classification through reliability-weighted cross-attention fusion and (ii) reference-based evidence generation. For the latter, query-reference channel similarities and a patch-level saliency map guide a frozen Vision Language Model (VLM) through an Observe–Generate–Refine loop, without forensic instruction tuning of the VLM. On the evaluated benchmark, 2CAP achieves overall AUC .963 and F1 .844; retouching F1 improves from .868 for the strongest compared baseline to .961. Across five VLM configurations, shared-parser paired evaluation shows model-dependent effects: evidence improves change-type accuracy over evidence-free refinement. These results support manipulation-type discrimination while delimiting the benefits of frozen-VLM refinement.
View details
