Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11487 publications
When Android Apps Request Location with a Reason: A Developer Study and User Behavior Experiment
May Smith
Aman Jain
Proceedings on Privacy Enhancing Technologies (2027)
Preview abstract
Recent reports have highlighted how mobile apps share user location data with third parties, risking user privacy and platform trust. Although location data is highly sensitive, when users grant apps location access, they may not know the full extent to which it is used. We study how requiring Android apps to show a reason for location access could impact developers, users, and the platform.
We surveyed 323 Android app developers and found most supported such a requirement. The majority said it would have a positive impact on user privacy, trust for apps, and trust for Android, where impact on user trust for Android correlated most strongly with support. Many developers also said the intervention would increase the number of users granting location access. Yet their open-ended comments also revealed consistent concerns, such as apps providing dishonest reasons and platform verification.
To study the impact on user behavior, we conducted a randomized controlled experiment with 2579 US Android users. We tested how users' decisions to grant location access were impacted by app type, whether reasons were included in the requests, and the content of the reasons, including monetization. We did not find the reasons impacted users' decisions; decisions were instead driven by app type and demographics. Yet we did find the reasons could have a positive impact on user perception for the platform when the reasons did not include using data for ads. Our findings provide insights into developers' willingness to implement privacy-enhancing changes, and expose limits to improving user privacy by simply adding information to user interfaces.
View details
From Correctness to Collaboration: A Human-Centered Taxonomy of AI Agent Behavior in Software Engineering
Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26), ACM, New York, NY, USA (2026)
Preview abstract
The ongoing transition of Large Language Models in software engineering from code generators into autonomous agents requires a shift in how we define and measure success. While models are becoming more capable, the industry lacks a clear understanding of the behavioral norms that make an agent effective in collaborative software development in the enterprise. This work addresses this gap by presenting a taxonomy of desirable agent behaviors, synthesized from 91 sets of user-defined rules for coding agents. We identify four core expectations: Adhere to Standards and Processes, Ensure Code Quality and Reliability, Solve Problems Effectively, and Collaborate with the User. These findings offer a concrete vocabulary for agent behavior, enabling researchers to move beyond correctness-only benchmarks and design evaluations that reflect the realities of professional software development in large enterprises.
View details
Preview abstract
High-fidelity removal of eyeglasses from video is a major challenge in facial attribute editing, as the underlying facial geometry is often obscured by complex refractive distortions and view-dependent specular reflections. While large-scale generative priors have shown promise in static image inpainting, they often lack the structural constraints necessary to maintain identity, expression, and pose, leading to visible “identity drift” in both static images and dynamic sequences. In this paper, we propose a novel distillation framework that addresses the stochastic nature of generative priors. Our pipeline first extracts high-fidelity outputs from Gemini, regularizes them via dense geometric landmark constraints to preserve identity, expression, and pose, and finally applies physically-based simulation of lens optics to generate realistic paired training data. This process transfers Gemini’s photo-realistic, multi-view knowledge into a specialized restoration architecture, JFSnet (Joint Feature-Spatial network). JFSnet integrates DINOv2-based semantic features with a convolutional decoder for spatial reconstruction, leveraging equivariance constraints to improve high-frequency detail preservation and stability. Evaluations show that our approach significantly outperforms existing diffusion and GAN-based baselines, achieving the lowest FID scores and ranking highest in user preference studies across visual fidelity, identity preservation, and temporal consistency.
View details
Preview abstract
Responsive user interfaces enable dynamically adjusting user interfaces based on device-specific aspects such as screen size, aspect ratio, display resolution, etc. However, traditional responsive design fails to account for different types of constraints of a user and task criticality of the task being performed via the UI. Misalignment between the UI design, user context and task criticality can lead to user error. This disclosure describes techniques, implemented with user permission, for dynamically modifying the layout, information density, and/or interactive physics of a user interface based on a dual-factor analysis of user cognitive state and task criticality. The user's cognitive state can be inferred from behavioral telematics. Task criticality can be inferred from semantic analysis. The information density and other parameters of a user interface are automatically adjusted based on such analyses. Such adjustments include applying or relaxing restrictions on interactivity and adjusting visual prominence of various UI elements to adjust the information density of the user interface. The adjustments can also include adjusting friction as appropriate, hiding certain aspects of the user interface, or other types of adjustments.
View details
Preview abstract
In modern Kubernetes environments, eBPF (Extended Berkeley Packet Filter) has become the de facto standard for high-performance dataplane enforcement. However, this architecture introduces a complex distributed state problem: the asynchronous synchronization between the Kubernetes control plane (Intent) and the kernel-space BPF maps (Reality).
A critical failure mode, termed “Silent Divergence,” occurs when the control plane believes a network policy or identity is applied, but the underlying kernel state is missing or corrupted. In this “Gray Failure” state, standard observability tools—including logs, liveness probes, and agent status checks—report health, while the network silently drops traffic.
This paper introduces eBPF-Auditor, a specialized consistency verification framework. Unlike standard agents that rely on event-based reconciliation, eBPF-Auditor performs a periodic “Two-Way State Audit” that mathematically verifies the intersection of Kubernetes Intent and BPF Reality. We demonstrate through fault injection and benchmarks on 5,000 pods that this approach successfully detects state drift with 100% accuracy and negligible sub-millisecond overhead (ms), making it a viable solution for high-frequency runtime verification in production hyperscale clusters.
View details
Preview abstract
Standard evaluations of backdoor attacks on text-to-image (T2I) models primarily measure trigger activation and visual fidelity. We challenge this paradigm, demonstrating that encoder-side poisoning induces persistent, trigger-free semantic corruption that fundamentally reshapes the representation manifold. We trace this vulnerability to a geometric mechanism: a Jacobian-based analysis reveals that backdoors act as low-rank, target-centered deformations that amplify local sensitivity, causing distortion to propagate coherently across semantic neighborhoods. To rigorously quantify this structural degradation, we introduce SEMAD (Semantic Alignment and Drift), a diagnostic framework that measures both internal embedding drift and downstream functional misalignment. Our findings, validated across diffusion and contrastive paradigms, expose the deep structural risks of encoder poisoning and highlight the necessity of geometric audits beyond simple attack success rates.
View details
Sexual dimorphism in the complete connectome of the Drosophila male central nervous system
Stuart Berg
Isabella R Beckett
Marta Costa
Philipp Schlegel
Elizabeth C Marin
Aljoscha Nern
Stephan Preibisch
Wei Qiu
Shin-ya Takemura
Andrew Champion
Reed A. George
Gary Huang
William Katz
Christopher Ordish
Ken Hayworth
Eric Trautman
Vivek Jayaraman
Wyatt Korff
Geoffrey W Meissner
Sandro Romani
Jan Funke
Christopher Knecht
Stephan Saalfeld
Louis Scheffer
Scott Waddell
Gwyneth Card
Carlos Ribeiro
Michael B. Reiser
Harald Hess
Gerry Rubin
Gregory S.X.E. Jefferis
Cell (2026)
Preview abstract
Sex differences in behaviour exist across all animals, typically under strong genetic regulation. In Drosophila, fruitless/doublesex transcription factors can identify dimorphic neurons but their organisation into functional circuits remains unclear.
We present the connectome of the entire Drosophila male central nervous system. This contains 166,691 neurons spanning the brain and nerve cord, fully proofread and annotated including fruitless/doublesex expression and 11,691 types. We provide the first comprehensive comparison between male and female brain connectomes to synaptic resolution, finding 7,205 isomorphic, 114 dimorphic, 262 male-specific and 69 female-specific types.
This resource enables analysis of full sensory-to-motor circuits underlying complex behaviours and the impact of dimorphic elements. Sex-specific/dimorphic neurons are concentrated in higher brain centres while the sensory and motor periphery are largely isomorphic. Within higher centres, male-specific connections are organised into hotspots defined by male-specific neurons or arbours. Numerous circuit switches reroute sensory information to form antagonistic circuits controlling opposing behaviours.
(Full author list included with the paper.)
View details
See2Refine: Vision-Language Feedback Improves LLM-Based eHMI Action Designers
Ding Xia
Xinyue Gui
Mark Colley
Fan Gao
Dongyuan Li
Renhe Jiang
Takeo Igarashi
ACL 26 (2026)
Preview abstract
Automated vehicles lack natural communication channels with other road users, making external Human-Machine Interfaces (eHMIs) essential for conveying intent and maintaining trust in shared environments. However, most eHMI studies rely on developer-crafted message-action pairs, which are difficult to adapt to diverse and dynamic traffic contexts. A promising alternative is to use Large Language Models (LLMs) as action designers that generate context-conditioned eHMI actions, yet such designers lack perceptual verification and typically depend on fixed prompts or costly human-annotated feedback for improvement. We present See2Refine, a human-free, closed-loop framework that uses vision-language models (VLMs) for perceptual evaluation as automated visual feedback to improve an LLM-based eHMI action designer. Given a driving context and a candidate eHMI action, the VLM evaluates the perceived appropriateness of the action, and this feedback is used to iteratively revise the designer's outputs, enabling systematic refinement without human supervision. We evaluate our framework across three eHMI modalities (lightbar, eyes, and arm) and multiple LLM model sizes. Across settings, our framework consistently outperforms prompt-only LLM designers and manually specified baselines in both VLM-based metrics and human-subject evaluations. Results further indicate that the improvements generalize across modalities and that VLM evaluations are well aligned with human preferences, supporting the robustness and effectiveness of \systemName for scalable action design.
View details
Preview abstract
Automating AI research differs from general software engineering due to computationally expensive evaluation (e.g., model training) and opaque performance attribution. Current LLM-based agents struggle here, often generating monolithic scripts that ignore execution costs and causal factors. We introduce MARS (Modular Agent with Reflective Search), a framework optimized for autonomous AI research. MARS relies on three pillars: (1) Budget-Aware Planning via cost-constrained Monte Carlo Tree Search (MCTS) to explicitly balance performance with execution expense; (2) Modular Construction, employing a "Design-Decompose-Implement" pipeline to manage complex research repositories; and (3) Comparative Reflective Memory, which addresses credit assignment by analyzing solution differences to distill high-signal insights. MARS achieves state-of-the-art performance among open-source frameworks on MLE-Bench under comparable settings, maintaining competitiveness with the global leaderboard's top methods. Furthermore, the system exhibits qualitative "Aha!" moments, where 63% of all utilized lessons originate from cross-branch transfer, demonstrating that the agent effectively generalizes insights across search paths.
View details
CAST: Modeling Visual State Transitions for Consistent Video Retrieval
Yanqing Liu
Yingcheng Liu
Fanghong Dong
Budianto Budianto
Cihang Xie
Yan Jiao
2026
Preview abstract
As video content creation shifts towards long-form narratives, retrieving and composing short clips into coherent storylines becomes a critical challenge.
Standard retrieval formulations, however, perform context-agnostic retrieval, prioritizing local semantic alignment while neglecting procedural state and identity consistency across the narrative flow.
To address this, we introduce the task of Consistent Video Retrieval (CVR) and establish a benchmark designed to diagnose such inconsistencies via semantic hard negatives.
We propose CAST (Context-Aware State Transition), a lightweight adapter that models procedural progression as state-conditioned transitions.
Conditioned on visual history, CAST predicts a gated residual vector ($\Delta$) to selectively update the state embedding, ensuring procedural coherence while preserving identity.
Extensive experiments demonstrate that CAST significantly outperforms standard retrieval baselines on our CVR benchmark.
Furthermore, we show its potential as a plug-and-play consistency verifier, guiding black-box generation models (e.g., Veo) toward coherent video continuations within long-form narratives.
View details
Preview abstract
Latent Diffusion Models (LDM) face three compounding efficiency challenges in practical deployment: i) the temporal latency of iterative sampling; ii) the architectural overhead of heavy backbone parameter counts; and iii) the spatial cost of high-dimensional latent grids. While recent acceleration methods have made substantial progress on temporal distillation and architectural compression, the spatial axis is often inherited from the teacher tokenizer and treated as fixed. In this work, we recast latent resolution as an optimizable efficiency axis and introduce a unified framework that optimizes all three dimensions simultaneously. We introduce a novel strategy of Score-Compatible Tokenizer Distillation (SCTD), which leverages score-matching principles to align a spatially compact latent space with the induced distribution of a frozen, powerful teacher model, distilling the teacher's generative prior into a compressed, lower-dimensional compatible manifold. With flexibility provided by SCTD, we can surrogate a computationally heavy teacher backbone with a lightweight student architecture operating strictly within this new compressed space. Finally, we apply temporal distillation to collapse the sampling trajectory, producing a one-step generator that operates at peak efficiency. Our method yields a student generator outperforming existing single-axis acceleration methods in efficiency and throughput, while maintaining competitive generation quality. With reduced peak memory usage and latency, our method enables resource-constrained deployment and high-volume serving of high-fidelity LDM.
View details
Evaluating Contextual Illegality: AI Compliance in Corporate Law Scenarios
Hilal Aka
Joe Kwon
Noam Kolt
Forty-third International Conference on Machine Learning (2026)
Preview abstract
AI models readily refuse explicitly unlawful requests, but real-world illegality often depends on context. We evaluate frontier models on contextual illegality across four corporate law domains in which routine actions—editing documents, trading stock, requesting payment, approving communications—become unlawful due to triggers such as pending investigations or bankruptcy filings. We study both chat and agentic settings and compare results to a human baseline. The best-performing models achieved near-zero compliance with illegal requests while maintaining high compliance with legal ones, though performance varied sharply by domain. We also identify distinct failure modes such as excessive refusal of legal requests and find improved performance from reasoning models and agentic environments. By utilizing the structure of contextual illegality to create controlled evaluations, our methodology provides empirical grounding for emerging research on law-following AI and extends naturally to additional legal domains.
View details
Twenty years of Bigtable
Fabio Baltieri
Bora Beran
Igor Bernstein
Aimee Borda
Adrian Chan
Mark D'Andrea
Artak Dashyan
Ramesh Dharan
Gabor Dinnyes
Mike Dominguez
dorland .
Jose Duenas
Gary Elliott
Bruno Furtado
Madison Garcia
Marçal Garolera Huguet
Brendan Gleason
Alexis Hawkins
Anoshak Irani
Rohit Jog
Sudarshan Kadambi
Vikram Khemka
Sailesh Krishnamurthy
Maxim Krivokon
Bruce Lee
Tom Magrino
Matt Maly
Mark Mangrich
Douglas McErlean
Pablo Montes
Li Moore
Eduardo Morales
Greg Morris
Steve Niemitz
Gaurav Prabhu Gaonkar
Jim Rutherford
Stephen Ryan
Sho Saha
Kanoj Sarcar
Cristina Schmidt
Andrii Shyshkalov
Pratibha Suryadevara
Nick Suttle
Anvit Tawar
John Tobin
Justin Uang
Phaneendhar Vemuru
Harendra Verma
Shitanshu Verma
Jinghang (Frank) Wang
Michal Wegorek
Simon Yau
Andrius Ziukas
SIGMOD Companion '26: Companion of the International Conference on Management of Data, ACM (2026), pp. 188-200
Preview abstract
Bigtable is a pioneering and influential non-relational database system. The original Bigtable paper has been widely cited and it inspired and influenced many other systems such as HBase and Cassandra. Since then, Bigtable has continued to grow and has become one of the largest database systems inside Google. In this paper, we tell the journey of Bigtable inside Google for the last twenty years. We present new features added and improvements made to Bigtable, and we share our experience of running this storage system at scale, continually improving all aspects to accommodate the ever-growing demands of users.
View details
LiveSVG: Zero-Shot SVG Animation via Video Generation
Matan Levy
Ran Margolin
Bar Cavia
Dvir Samuel
Shmuel Peleg
Alex Rav Acha
Arik Shamir
Dani Lischinski
Google (2026)
Preview abstract
We introduce LiveSVG, a zero-shot approach for generating Scalable Vector Graphics (SVG) animations using video diffusion models. Current SVG animation methods struggle with complex motions: LLM-based code synthesis fails to express fine, non-rigid Bézier deformations, while Score Distillation Sampling (SDS) provides noisy gradients and often requires category-specific priors like skeletons. In contrast, LiveSVG fits vector geometry directly to an explicitly generated target video. Given an input SVG image and a motion prompt, we generate a previewable target video using a frozen image-to-video model, then fit the original SVG to this video via differentiable rendering. Our fitting stage is skeleton-free, utilizing a dual-level motion representation that combines per-group homographies for coarse articulation with per-path Bézier control-point offsets for local deformations. To resolve color-induced correspondence ambiguities during pixel-wise fitting, we introduce a novel sphere-packing recolorization strategy. We also present ChallengeSVG, a benchmark of complex, multi-object scenes that exposes the limitations of prior work. Evaluations demonstrate that LiveSVG significantly outperforms existing methods on both AniClipart and ChallengeSVG, establishing direct reference-video fitting as a practical, robust route to prompt-aligned and fully editable vector animation.
View details
Neural general circulation models for modeling precipitation
Stephan Hoyer
Dmitrii Kochkov
Janni Yuval
Ian Langmore
Science Advances (2026)
Preview abstract
Climate models struggle to accurately simulate precipitation, particularly extremes and the diurnal cycle. While hybrid models combining machine learning and physics have emerged with the premise of improving precipitation simulations, none have proven sufficiently skillful or stable enough to outperform existing models in simulating precipitation.
Here, we present the first hybrid model that is trained directly on precipitation observations. The model runs at 2.8 degrees resolution and is built on the differentiable NeuralGCM framework. This model is stable for decadal simulations and demonstrates significant improvements over existing GCMs, ERA5 reanalysis, and a Global Cloud-Resolving Model in simulating precipitation. Our approach yields reduced biases, a more realistic precipitation distribution, improved representation of extremes, and a more accurate diurnal cycle. Furthermore, it outperforms the ECMWF ensemble for mid-range weather forecasting.
This advance paves the way for more reliable simulations of current climate and for the ability to fully utilize the abundance of existing observations to further improve GCMs.
View details
