Publications
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Our teams aspire to make discoveries that impact everyone, and core to our approach is sharing our research and tools to fuel progress in the field.
Sort By
1 - 15 of 11411 publications
Tech Worker Challenges Managing Humanlike GenAI
Eric Corbett
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, ACM (2026), pp. 1-18
Preview abstract
Organizations are adopting or exploring anthropomorphic genAI — meaning XYZ. Anthropomorphic AI is often held up for its potential to improve the productivity and efficiency of workers and technologies; however, there are not yet accepted industry-wide standards for the responsible development of anthropomorphic technologies. Given their roles as central figures responsible for implementing anthropomorphic genAI into technologies that are served to the broader public, we must understand workers’ reasoning about anthropomorphic genAI to understand its impacts. However, there is a dearth of empirical knowledge about technology workers’ perspectives on anthropomorphic technologies, including their perspectives on potential risks and benefits. To address this gap, we conducted focus groups with 31 technology workers across 6 job roles (UX, software engineers, product managers, designers, marketing, and trust and safety) regarding how they define anthropomorphic genAI, their perceptions of anthropomorphic genAI, and their experiences working with anthropomorphic genAI. We find that workers’ have expansive definitions of what constitutes “humanlike” AI, which at times sit in tension with each other. They draw on their personal and professional standpoints to sensemake about real and possible anthropomorphic genAI hazards to people, knowledge work fields, and society at-large. Importantly, we find that these social hazards map to different facets of anthropomorphic genAI, suggesting that effective mitigation of personal and social risks requires developer attention to specific dimensions of anthropomorphism. We mapped the relationships between dimensions of anthropomorphism and hazards, to support technology workers. We argue that effective mitigation of the risks of anthropomorphism requires attention to the multiple facets of anthropomorphism.
View details
Platform Competition in the Autobidding World
Ariel Schvartzman
Andres Perlroth
Mingfei Zhao
Proceedings of the ACM Web Conference 2026, Association for Computing Machinery, New York, NY, USA, 87–98
Preview abstract
We study the problem of auction design for advertising platforms that face strategic advertisers who are bidding across platforms. Each advertiser's goal is to maximize their total value or conversions while satisfying some constraint(s) across all the platforms they participates in. In this paper, we focus on advertisers with return-over-investment (henceforth, ROI) constraints, i.e. each advertiser is trying to maximize value while making sure that their ROI across all platforms is no less than some target value. An advertiser interacts with the platforms through autobidders -- for each platform, the advertiser strategically chooses a target ROI to report to the platform's autobidder, which in turn uses a uniform bid multiplier to bid on the advertiser's behalf on the queries owned by the given platform.
Our main result is an auction comparator theorem that can be used to choose between different auction mechanisms by a revenue-maximizing platform operating in a competitive environment. Analogous to the Linkage Principle for interdependent values, our comparator demonstrates that, for a broad class of advertiser valuations, second-price auctions may sometimes generate higher revenue than first-price auctions. This contrasts sharply with the single-platform setting, where first-price auctions are optimal for both revenue and welfare. Our findings highlight the critical importance of considering inter-platform competition when designing optimal auctions.
View details
Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI
Xinyue Gui
Ding Xia
Mark Colley
Yuan Li
Vishal Chauhan
Anubhav Anubhav
Ehsan Javanmardi
Stela Hanbyeol Seo
Chia-Ming Chang
Manabu Tsukada
Takeo Igarashi
Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI 26)
Preview abstract
Field studies are irreplaceable but costly, time-consuming, and error-prone, which need careful preparation. Inspired by rapid-prototyping in manufacturing, we propose a fast, low-cost evaluation method using Vision-Language Model (VLM) personas to simulate outcomes comparable to field results. While LLMs show human-like reasoning and language capabilities, autonomous vehicle (AV)-pedestrian interaction requires spatial awareness, emotional empathy, and behavioral generation. This raises our research question: To what extent can VLM personas mimic human responses in field studies? We conducted parallel studies: 1) one real-world study with 20 participants, and 2) one video-study using 20 VLM personas, both on a street-crossing task. We compared their responses and interviewed five HCI researchers on potential applications. Results show that VLM personas mimic human response patterns (e.g., average crossing times of 5.25 s vs. 5.07 s) lack the behavioral variability and depth. They show promise for formative studies, field study preparation, and human data augmentation.
View details
Preview abstract
As the ECMAScript specification evolves, industrial-scale JavaScript compilers face the challenge of supporting modern language syntax while maintaining compatibility for diverse execution environments. Traditionally, compilers solve this by running transpilation passes in a monolithic pipeline, where the transpilation passes are chosen to execute strictly based on a target language level. This results in significant computational waste, as compilers perform expensive Abstract Syntax Tree (AST) traversals to lower features that may not exist in the actual input source code. We present a static analysis improvement that conditionally executes transpiler passes based on accurately tracking and dynamically maintaining the exact set of language features seen in the compilation unit throughout the transpilation process. It is implemented in the production Google Closure Compiler. By populating and maintaining a FeatureSet at every JavaScript script-level, it dynamically skips running the unnecessary lowering passes. We detail the architectural safeguards - including strategic pass ordering and dynamic validation of the transpiled code for feature-correctness. Evaluation of this improvement on large-scale production applications produced a considerable reduction in compilation time and saved compute and memory usage.
View details
Preview abstract
Optimizing large-language model (LLM) training and serving on large-sacle distributed systems with hundreds and thousands of accelerators is always a challenging task due to the fast evloving LLMs, strong domain expertise required, and various optimization goals from different worklaods. Existing methods rely on either handcrafted optimization performed by human experts, which is tedious and time-consuming or resource-intensive black-box searches, which lack the extensibility to keep pace with evolving models and hardware. To address this, we introduce PROMPTS, a novel multi-agent framework that complements traditional search methods with expert-informed reasoning. It automates the diagnosis of performance bottlenecks by synthesizing profiler data and leverages a knowledge base to propose optimized sharding configurations with detailed justifications.
Across eight real-world production workloads, PROMPTS demonstrated remarkable efficiency and accuracy, delivering performance improvements of up to 434%. These workloads spanned diverse model architectures, hardware platforms, computational scales, and various stages of the machine learning lifecycle (pre-training, serving, and post-training). In every case, the configuration adopted by human engineers was identified within the agent's top three proposals from a single invocation. Furthermore, the agent's top-ranked recommendation was the one ultimately adopted in 87.5% of cases, showcasing its ability to not only find optimized solutions, but also to correctly prioritize them. Our work establishes PROMPTS as a scalable, extensible, and explainable methodology for AI-assisted performance engineering in large-scale ML systems.
View details
MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR
Sieun Kim
Qianhui Zheng
Ruoyu Xu
Ravi Tejasvi
Anuva Kulkarni
Junyi Zhu
2026
Preview abstract
In Extended Reality (XR), complex acoustic environments often overwhelm users, compromising both scene awareness and social engagement due to entangled sound sources. We introduce MoXaRt, a real-time XR system that uses audio-visual cues to separate these sources and enable fine-grained sound interaction. MoXaRt's core is a cascaded architecture that performs coarse, audio-only separation in parallel with visual detection of sources (e.g. faces, instruments). These visual anchors then guide refinement networks to isolate individual sources, separating complex mixes of up to five concurrent sources (e.g. two voices + three instruments) with ca. 2 second processing latency. We validate MoXaRt through a technical evaluation on a new, complex dataset we collected, and a 22-participant user study. Our results demonstrate that MoXaRt significantly improves communication clarity—boosting listening comprehension in noisy conditions by 33.2% (p=0.0058)—and significantly reduces cognitive load (M=7.50 vs. M=3.36, p<0.001), paving the way for more perceptive and socially adept XR experiences.
View details
Preview abstract
Online video platforms face an exponential challenge
in detecting and mitigating the flood of AI-generated “slop”
and synthetic spam perpetuated by coordinated malicious actors.
This content is increasingly designed to exploit the limitations
of traditional media forensics, often utilizing generative AI to
produce unique, localized variations of harmful or low-quality
material at scale. Traditional content-centric moderation fails
against this coordinated, adversarial generation strategy.
This paper presents a novel, scalable detection and classifi-
cation framework designed for online video platforms (OVP) to
identify and triage clusters of coordinated accounts exhibiting
a prevalence of adversarial synthetic content. The approach
leverages a multi-faceted architecture incorporating two core
machine learning components: a robust Coordinated Bot-Net
Detector (via Account Relatedness) and a Synthetic Pattern Clas-
sifier. Crucially, we introduce an advanced AI enhancement layer
utilizing Large Language Models (LLMs), specialized via Low-
Rank Adaptation (LoRA) and Automatic Prompt Optimization
(APO), to achieve rapid, high-precision semantic understanding
of emerging synthetic spam trends.
Evaluated across a representative evaluation dataset (N =
16, 250 weekly candidate channels across six major synthetic
abuse verticals), the system demonstrates high precision (FPR <
0.05%) in identifying coordinated synthetic spam networks.
Furthermore, the LLM-driven classification achieves a 74%
automated triage routing rate, saving over 1, 100 operational
review hours per week while reducing investigation turnaround
times by up to 50% (p < 0.001). This work details a critical
system design that provides essential scalability and adversarial
resilience against sophisticated generative attacks.
View details
Preview abstract
Multimodal large language models (LLMs) integrate and process information from multiple modalities such as text, images, audio, and video, enabling complex tasks such as audio translation and visual question answering. While powerful, this complexity introduces novel vulnerabilities to sophisticated adversarial attacks. This survey paper provides a comprehensive overview of this rapidly expanding field, systematically categorizing attacks that range from manipulations of single modalities (e.g., perturbed images or audio) to those exploiting cross-modal interactions. We overview how these attacks exploit weaknesses in model fusion, attention mechanisms, and representation learning and provided analyses on their potential for real-world consequences.
View details
Preview abstract
This framework manages AI agents by establishing behavioral boundaries and a persistent identity. It uses a multi-layered stack, combining safety rules with brand guidelines, to shape an agent's reasoning. Features include authority decay to limit power if confidence drops and memory segmentation to prevent data tampering. Centralized oversight ensures these digital representatives remain aligned with company policies through continuous monitoring and testing.
View details
Who is At-Risk? Surveying the Prevalence of Risk Factors and Tech-Facilitated Attacks in the United States
Sharon Heung
Claire Weizenegger
Mo Houtti
Tara Matthews
Ashley Walker
2026
Preview abstract
A growing body of qualitative research has identified contextual risk factors that elevate people’s chances of experiencing digital-safety attacks. However, the lack of quantitative data on the population level distribution of these risk factors prevents policymakers and tech companies from developing targeted, evidence-based interventions to improve digital safety. To address this gap, we surveyed 5,001 adults in the United States to analyze: (1) the frequency of and relationship between digital-safety attacks (e.g., scams, harassment, account hacking), and (2) how these attacks align with 10 contextual risk factors. Nearly half of our respondents identify as resource constrained, which significantly correlates with higher likelihood of experiencing four common attacks. We also present qualitative insights to expand our understanding of the factors beyond the existing literature (e.g., “prominence” included high-visibility roles in local communities). This study provides the first large-scale quantitative analysis correlating digital-safety attacks with contextual risk factors and demographics.
View details
Vibe Coding XR: Accelerating AI + XR Prototyping with XR Blocks and Gemini
Benjamin Hersh
Jiahao Ren
Xingyue Chen
Robert Timothy Bettridge
Faraz Faruqi
Anthony 'Xiang' Chen
Steve Toh
Google XR, Google (2026)
Preview abstract
While large language models have accelerated software development through "vibe coding", prototyping intelligent Extended Reality (XR) experiences remains inaccessible due to the friction of complex game engines and low-level sensor integration. To bridge this gap, we contribute XR Blocks, an open-source, modular WebXR framework that abstracts spatial computing complexities into high-level, human-centered primitives. Building upon this foundation, we present Vibe Coding XR, an end-to-end rapid prototyping workflow that leverages LLMs to translate natural language intent directly into functional XR software. Using a web-based interface, creators can transform high-level prompts (e.g., "create a dandelion that reacts to hand") into interactive WebXR applications in under a minute. We provide a preliminary technical evaluation on a pilot dataset (VCXR60) alongside diverse application scenarios highlighting mixed-reality realism, multi-modal interaction, and generative AI integrations. By democratizing spatial software creation, this work empowers practitioners to bypass low-level hurdles and rapidly move from "idea to reality." Code and live demos are available at https://xrblocks.github.io/gem and https://github.com/google/xrblocks.
View details
A Theoretical Framework for Modular Learning of Robust Generative Models
Proceedings of the 43rd International Conference on Machine Learning (ICML 2026)
Preview abstract
Training large-scale generative models is resource-intensive and relies heavily on heuristic dataset weighting. We address two fundamental questions: Can we train Large Language Models (LLMs) modularly, combining small, domain-specific experts to match monolithic performance, and can we do so robustly for any data mixture, eliminating heuristic tuning? We present a theoretical framework for modular generative modeling where a set of pre-trained experts are combined via a gating mechanism. We define the space of normalized gating functions $\mathcal{G}_{1}$ and formulate the problem as a minimax game to find a single robust gate that minimizes divergence to the worst-case data mixture. We prove the existence of such a robust gate using Kakutani's fixed-point theorem and show that modularity acts as a strong regularizer, with generalization bounds scaling with the lightweight gate's complexity. Furthermore, we prove that this modular approach can theoretically outperform models retrained on aggregate data, with the gap characterized by the Jensen-Shannon Divergence. Finally, we introduce a scalable Stochastic Primal-Dual algorithm and a Structural Distillation method for efficient inference. Empirical results on synthetic and real-world datasets confirm that our modular architecture effectively mitigates gradient conflict and can robustly outperform monolithic baselines.
View details
Preview abstract
Standard evaluations of backdoor attacks on text-to-image (T2I) models primarily measure trigger activation and visual fidelity. We challenge this paradigm, demonstrating that encoder-side poisoning induces persistent, trigger-free semantic corruption that fundamentally reshapes the representation manifold. We trace this vulnerability to a geometric mechanism: a Jacobian-based analysis reveals that backdoors act as low-rank, target-centered deformations that amplify local sensitivity, causing distortion to propagate coherently across semantic neighborhoods. To rigorously quantify this structural degradation, we introduce SEMAD (Semantic Alignment and Drift), a diagnostic framework that measures both internal embedding drift and downstream functional misalignment. Our findings, validated across diffusion and contrastive paradigms, expose the deep structural risks of encoder poisoning and highlight the necessity of geometric audits beyond simple attack success rates.
View details
Understanding U.S. Users' Security and Privacy Transparency Needs for Consumer-Facing Generative AI
Jiaxun Cao
Yu Dong
Chunxi Zhan
Rithvik Neti
Pardis Emami-Naeini
USENIX Symposium on Usable Privacy and Security (SOUPS) (2026)
Preview abstract
Users increasingly rely on consumer-facing generative AI (GenAI) for tasks ranging from everyday needs to sensitive use cases. Yet, it remains unclear whether and how existing security and privacy (S&P) communications in GenAI tools shape users’ adoption decisions and experiences. Understanding how users seek, interpret, and evaluate S&P information is critical for designing usable transparency that users can trust and act on. We conducted semi-structured interviews and design sessions with 21 U.S. GenAI users. Our findings suggest that available S&P information rarely drove initial adoption in practice, as participants often perceived it as incomplete, ineffective, or not credible. Instead, they relied on rough proxies (e.g., popularity) to infer S&P practices. After adoption, S&P uncertainty constrained participants’ willingness to use GenAI tools, especially for high-stakes purposes, and, in some cases, contributed to discontinued use. Participants therefore called for transparency that supports decisions and actions through trustworthy information (e.g., independent evaluations) and usable interfaces (e.g., on-demand disclosure). We categorize participants’ desired design practices into five dimensions to facilitate systematic future investigation into best practices. We conclude with recommendations for researchers, designers, and policymakers to improve S&P transparency in consumer-facing GenAI.
View details
