Google at ICLR 2025

Sat, Apr 26 | 2:00PM — 3:00PM, Hall 1 Apex

Open-Endedness, World Models, and the Automation of Innovation

Speaker: Tim Rocktäschel

Link to session

Accelerating Neural Network Training: An Analysis of the AlgoPerf Competition
Priya Kasimbeg, Frank Schneider, Runa Eschenhagen, Juhan Bae, Chandramouli Shama Sastry, Mark Saroufim, Boyuan Feng, Less Wright, Edward Z. Yang, Zachary Nado, Sourabh Medapati, Philipp Hennig, Michael Rabbat, George E. Dahl

Block Verification Accelerates Speculative Decoding
Ziteng Sun, Uri Mendlovic, Yaniv Leviathan, Asaf Aharoni, Jae Hun Ro, Ahmad Beirami, Ananda Theertha Suresh

Forte: Finding Outliers with Representation Typicality Estimation
Debargha Ganguly, Warren Morningstar, Andrew Yu, Vipin Chaudhary

Improving Language Model Distillation Through Hidden State Matching
Sayantan Dasgupta, Trevor Cohn

Improving Large Language Model Planning With Action Sequence Similarity
Xinran Zhao*, Hanie Sedghi, Bernd Bohnet, Dale Schuurmans, Azade Nova

MELODI: Exploring Memory Compression for Long Contexts
Yinpeng Chen, DeLesley Hutchins, Aren Jansen, Andrey Zhmoginov, David Racz, Jesper Andersen

Optimistic Games for Combinatorial Bayesian Optimization With Application to Protein Design
Melis Ilayda Bal, Pier Giuseppe Sessa, Mojmír Mutný, Andreas Krause

PFGuard: A Generative Framework With Privacy and Fairness Safeguards
Soyeon Kim, Yuji Roh, Geon Heo, Steven Euijong Whang

Privacy Auditing of Large Language Models
Ashwinee Panda, Xinyu Tang, Milad Nasr, Christopher A. Choquette-Choo, Prateek Mittal

ReCogLab: A Framework Testing Relational Reasoning & Cognitive Hypotheses on LLMs
Andrew Liu, Henry Prior, Gargi Balasubramaniam, Rivka Moroshko, Amir Zait, Ilia Labzovsky, Danny Karmon, Ishita Dasgupta, Kim Stachenfeld, Kenneth Marino

RRM: Robust Reward Model Training Mitigates Reward Hacking
Tianqi Liu, Wei Xiong*, Jie Ren, Lichang Chen*, Junru Wu, Rishabh Joshi, Yang Gao, Jiaming Shen, Zhen Qin, Tianhe Yu, Daniel Sohn, Anastasia Makarova, Jeremiah Liu, Yuan Liu, Bilal Piot, Abe Ittycheriah, Aviral Kumar, Mohammad Saleh

Selective Unlearning via Representation Erasure Using Domain Adversarial Training
Nazanin Mohammadi Sepahvand, Eleni Triantafillou, Hugo Larochelle, Doina Precup, James J. Clark, Daniel M. Roy, Gintare Karolina Dziugaite

Speculative RAG: Enhancing Retrieval Augmented Generation Through Drafting (see blog post)
Zilong Wang*, Zifeng Wang, Long T. Le, Huaixiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, Chen-Yu Lee, Tomas Pfister

SVG: 3D Stereoscopic Video Generation via Denoising Frame Matrix
Peng Dai*, Feitong Tan, Qiangeng Xu, David Futschik, Ruofei Du, Sean Fanello, Xiaojuan Qi, Yinda Zhang

Training LLMs Over Neurally Compressed Text
Brian Lester, Jaehoon Lee*, Alex Alemi*, Jeffrey Pennington, Adam Roberts, Jascha Sohl-Dickstein*, Noah Constant

Transformers Struggle to Learn to Search
Abulhair Saparov, Srushti Ajay Pawar, Shreyas Pimpalgaonkar, Nitish Joshi, Richard Yuanzhe Pang, Vishakh Padmakumar, Seyed Mehran Kazemi, Najoung Kim, He He

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF
Shicong Cen, Jincheng Mei, Katayoon Goshvadi, Hanjun Dai, Tong Yang, Sherry Yang, Dale Schuurmans, Yuejie Chi, Bo Dai

Variance-Reducing Couplings for Random Features
Isaac Reid, Stratis Markou, Krzysztof Choromanski, Richard E. Turner, Adrian Weller

YouTube-SL-25: A Large-Scale, Open-Domain Multilingual Sign Language Parallel Corpus
Garrett Tanzer, Biao Zhang

Agents' Room: Narrative Generation Through Multi-Step Collaboration
Fantine Huot, Reinald Kim Amplayo, Jennimaria Palomaki, Alice Shoshana Jakobovits, Elizabeth Clark, Mirella Lapata

Can We Talk Models Into Seeing the World Differently?
Paul Gavrikov, Jovita Lukasik, Steffen Jung, Robert Geirhos, M. Jehanzeb Mirza, Margret Keuper, Janis Keuper

Controlling Space and Time With Diffusion Models
Daniel Watson, Saurabh Saxena, Lala Li, Andrea Tagliasacchi, David Fleet

Deep MMD Gradient Flow Without Adversarial Training
Alexandre Galashov, Valentin De Bortoli, Arthur Gretton

Diffusion Models Are Real-Time Game Engines
Dani Valevski, Yaniv Leviathan, Moab Arar*, Shlomi Fruchter

Efficient Stagewise Pre-training via Progressive Subnetworks
Abhishek Panigrahi*, Nikunj Saunshi, Kaifeng Lyu*, Sobhan Miryoosefi, Sashank Reddi, Satyen Kale*, Sanjiv Kumar

ElasticTok: Adaptive Tokenization for Image and Video
Wilson Yan, Vlad Mnih, Aleksandra Faust, Matei Zaharia, Pieter Abbeel, Hao Liu

Fluid: Scaling Autoregressive Text-to-Image Generative Models With Continuous Tokens
Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, Yonglong Tian

Guided Score Identity Distillation for Data-Free One-Step Text-to-Image Generation
Mingyuan Zhou, Zhendong Wang, Huangjie Zheng, Hai Huang*

KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models
Eunice Yiu, Maan Qraitem, Anisa Noor Majhi, Charlie Wong, Yutong Bai, Shiry Ginosar, Alison Gopnik, Kate Saenko

Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG
Bowen Jin*, Jinsung Yoon, Jiawei Han, Sercan Ö. Arik

Measuring Non-Adversarial Reproduction of Training Data in Large Language Models
Michael Aerni, Javier Rando, Edoardo Debenedetti, Nicholas Carlini, Daphne Ippolito, Florian Tramèr

MUSE: Machine Unlearning Six-Way Evaluation for Language Models
Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, Chiyuan Zhang

OmnixR: Evaluating Omni-Modality Language Models on Reasoning Across Modalities
Lichang Chen, Hexiang Hu, Mingda Zhang, Yiwen Chen, Zifeng Wang, Yandong Li, Pranav Shyam, Tianyi Zhou, Heng Huang, Ming-Hsuan Yang, Boqing Gong

On Evaluating the Durability of Safeguards for Open-Weight LLMs
Xiangyu Qi, Boyi Wei, Nicholas Carlini, Yangsibo Huang, Tinghao Xie, Luxi He, Matthew Jagielski, Milad Nasr, Prateek Mittal, Peter Henderson

Revisit Micro-Batch Clipping: Adaptive Data Pruning via Gradient Manipulation
Lun Wang

Speculative Knowledge Distillation: Bridging the Teacher–Student Gap Through Interleaved Sampling
Wenda Xu*, Rujun Han, Zifeng Wang, Long T. Le, Dhruv Madeka, Lei Li, William Yang Wang, Rishabh Agarwal, Chen-Yu Lee, Tomas Pfister

Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning
Bahare Fatemi, Mehran Kazemi, Anton Tsitsulin, Karishma Malkan, Jinyeong Yim, John Palowitch, Sungyong Seo, Jonathan Halcrow, Bryan Perozzi

The Hyperfitting Phenomenon: Sharpening and Stabilizing LLMs for Open-Ended Text Generation
Fredrik Carlsson, Fangyu Liu, Daniel Ward, Murathan Kurfali, Joakim Nivre

When Does Compositional Structure Yield Compositional Generalization? A Kernel Theory
Samuel Lippl, Kim Stachenfeld

A Simple Approach to Unifying Diffusion-Based Conditional Generation
Xirui Li, Charles Herrmann, Kelvin C.K. Chan, Yinxiao Li, Deqing Sun, Chao Ma, Ming-Hsuan Yang

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
Zeman Li, Xinwei Zhang, Peilin Zhong, Yuan Deng, Meisam Razaviyayn, Vahab Mirrokni

AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, Daniel Toyama, Robert Berry, Divya Tyamagundlu, Timothy Lillicrap, Oriana Riva

CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL
Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Özcan, Sercan Ö. Arik

Diffusion Models and Gaussian Flow Matching: Two Sides of the Same Coin
Ruiqi Gao, Emiel Hoogeboom, Jonathan Heek, Valentin De Bortoli, Kevin Patrick Murphy, Tim Salimans

Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts?
Sravanti Addepalli, Yerram Varun, Arun Suggala, Karthikeyan Shanmugam, Prateek Jain

Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation
Xiaojuan Wang, Boyang Zhou, Brian Curless, Ira Kemelmacher-Shlizerman, Aleksander Holynski, Steven M. Seitz

Learning Continually by Spectral Regularization
Alex Lewandowski, Michał Bortkiewicz, Saurabh Kumar, András György, Dale Schuurmans, Mateusz Ostaszewski, Marlos C. Machado

LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations
Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, Yonatan Belinkov

Machine Unlearning Fails to Remove Data Poisoning Attacks
Martin Pawelczyk, Jimmy Z. Di, Yiwei Lu, Gautam Kamath, Ayush Sekhari, Seth Neel

Plastic Learning With Deep Fourier Features
Alex Lewandowski, Dale Schuurmans, Marlos C. Machado

Scalable Extraction of Training Data From Aligned, Production Language Models
Milad Nasr, Javier Rando, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Florian Tramèr, Katherine Lee

Scalable Influence and Fact Tracing for Large Language Model Pre-training
Tyler A. Chang*, Dheeraj Rajagopal, Tolga Bolukbasi, Lucas Dixon, Ian Tenney

Selective Attention Improves Transformer
Yaniv Leviathan, Matan Kalman, Yossi Matias

Sufficient Context: A New Lens on Retrieval Augmented Generation Systems
Hailey Joren*, Jianyi Zhang*, Chun-Sung Ferng, Da-Cheng Juan, Ankur Taly, Cyrus Rashtchian

TIPS: Text-Image Pretraining With Spatial Awareness
Kevis-Kokitsi Maninis, Kaifeng Chen, Soham Ghosh*, Arjun Karpur, Koert Chen, Ye Xia, Bingyi Cao, Daniel Salz, Guangxing Han, Jan Dlabal, Dan Gnanapragasam, Mojtaba Seyedhosseini, Howard Zhou, André Araujo

Towards Scalable Exact Machine Unlearning Using Parameter-Efficient Fine-Tuning
Somnath Basu Roy Chowdhury, Krzysztof Choromanski, Arijit Sehanobish, Avinava Dubey, Snigdha Chaturvedi

VideoPhy: Evaluating Physical Commonsense for Video Generation
Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, Aditya Grover

A Black Swan Hypothesis: The Role of Human Irrationality in AI Safety
Hyunin Lee, Chanwoo Park, David Abel, Ming Jin

CURIE: Evaluating LLMs on Multitask Scientific Long-Context Understanding and Reasoning (see blog post)
Hao Cui, Zahra Shamsi, Gowoon Cheon, Xuejian Ma, Shutong Li, Maria Tikhanovskaya*, Peter Norgaard, Nayantara Mudur, Martyna Plomecka*, Paul Raccuglia, Yasaman Bahri, Victor V. Albert, Pranesh Srinivasan, Haining Pan, Philippe Faist, Brian Rohr, Michael J. Statt, Dan Morris, Drew Purves, Elise Kleeman, Ruth Alcantara, Matthew Abraham, Muqthar Mohammad, Ean Phing VanLee, Chenfei Jiang, Elizabeth Dorfman, Eun-Ah Kim, Michael P. Brenner, Viren Jain, Sameera Ponda, Subhashini Venugopalan

DeciMamba: Exploring the Length Extrapolation Potential of Mamba
Assaf Ben-Kish, Itamar Zimerman, Shady Abu-Hussein, Nadav Cohen, Amir Globerson, Lior Wolf, Raja Giryes

Differentially Private Optimization for Non-Decomposable Objective Functions
Weiwei Kong, Andrés Muñoz Medina, Mónica Ribero

Diversity-Rewarded CFG Distillation
Geoffrey Cideron, Andrea Agostinelli, Johan Ferret, Sertan Girgin, Romuald Elie, Olivier Bachem, Sarah Perrin, Alexandre Ramé

Faster, More Efficient RLHF Through Off-Policy Asynchronous Learning
Michael Noukhovitch, Shengyi Huang, Sophie Xhonneux, Arian Hosseini, Rishabh Agarwal, Aaron Courville

LevAttention: Time, Space and Streaming Efficient Algorithm for Heavy Attentions
Ravindran Kannan, Chiranjib Bhattacharyya, Praneeth Kacham, David Woodruff*

Mitigating Object Hallucination in MLLMs via Data-Augmented Phrase-Level Alignment
Pritam Sarkar*, Sayna Ebrahimi, Ali Etemad*, Ahmad Beirami, Sercan Ö. Arik, Tomas Pfister

Motion Control of High-Dimensional Musculoskeletal Systems With Hierarchical Model-Based Planning
Yunyue Wei, Shanning Zhuang, Vincent Zhuang, Yanan Sui

Multi-Agent Cooperation Through Learning-Aware Policy Gradients
Alexander Meulemans, Seijin Kobayashi, Johannes Von Oswald, Nino Scherrer, Eric Elmoznino, Blake Aaron Richards, Guillaume Lajoie, Blaise Agüera y Arcas, João Sacramento

Near-Exact Privacy Amplification for Matrix Mechanisms
Christopher A. Choquette-Choo, Thomas Steinke, Abhradeep Guha Thakurta, Arun Ganesh, Saminul Haque

Offline Hierarchical Reinforcement Learning via Inverse Optimization
Carolin Schmidt, Daniele Gammelli, James Harrison, Marco Pavone, Filipe Rodrigues

SIMPL: Scalable and Hassle-Free Optimisation of Neural Representations From Behaviour
Tom M. George, Pierre Glaser, Kimberly Stachenfeld, Caswell Barry, Claudia Clopath

Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling
Hritik Bansal, Arian Hosseini, Rishabh Agarwal, Vinh Q. Tran, Mehran Kazemi

SMITE: Segment Me In TimE
Amirhossein Alimohammadi, Sauradip Nag, Saeid Asgari Taghanaki, Andrea Tagliasacchi, Ghassan Hamarneh, Ali Mahdavi Amiri

Steering Large Language Models Between Code Execution and Textual Reasoning
Yongchao Chen, Harsh Jhamtani, Srinagesh Sharma, Chuchu Fan, Chi Wang

The Crystal Ball Hypothesis in Diffusion Models: Anticipating Object Positions From Initial Noise
Yuanhao Ban, Ruochen Wang, Tianyi Zhou, Boqing Gong, Cho-Jui Hsieh, Minhao Cheng

The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD
Thomas Steinke, Milad Nasr, Arun Ganesh, Borja Balle, Christopher A. Choquette-Choo, Matthew Jagielski, Jamie Hayes, Abhradeep Guha Thakurta, Adam Smith, Andreas Terzis

Training-Free Diffusion Model Alignment With Sampling Demons
Po-Hung Yeh, Kuang-Huei Lee, Jun-Cheng Chen

Unlearn and Burn: Adversarial Machine Unlearning Requests Destroy Model Accuracy
Yangsibo Huang, Daogao Liu, Lynn Chua, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Milad Nasr, Amer Sinha, Chiyuan Zhang

A Unifying Framework for Representation Learning
Shaden Naif Alshammari, John Hershey, Axel Feldmann, William Freeman, Mark Hamilton

Achieving Dimension-Free Communication in Federated Learning via Zeroth-Order Optimization
Zhe Li, Bicheng Ying, Zidong Liu, Chaosheng Dong, Haibo Yang

Building Math Agents With Multi-Turn Iterative Preference Learning
Wei Xiong, Chengshuai Shi, Jiaming Shen, Aviv Rosenberg, Zhen Qin, Daniele Calandriello, Misha Khalman, Rishabh Joshi, Bilal Piot, Mohammad Saleh, Chi Jin, Tong Zhang, Tianqi Liu

Debiasing Federated Learning With Correlated Client Participation
Zhenyu Sun, Ziyang Zhang, Zheng Xu, Gauri Joshi, Pranay Sharma, Ermin Wei

Descent With Misaligned Gradients and Applications to Hidden Convexity
Aditya Bhaskara, Ashok Cutkosky, Ravi Kumar, Manish Purohit

Do LLMs Have Consistent Values?
Naama Rozen, Liat Bezalel, Gal Elidan, Amir Globerson, Ella Daniel

Efficient Exploration and Discriminative World Model Learning With an Object-Centric Abstraction
Anthony GX-Chen, Kenneth Marino, Rob Fergus

Fourier Head: Helping Large Language Models Learn Complex Probability Distributions
Nate Gillman, Daksh Aggarwal, Michael Freeman, Saurabh Singh, Chen Sun

Generative Verifiers: Reward Modeling as Next-Token Prediction
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, Rishabh Agarwal

Glauber Generative Model: Discrete Diffusion Models via Binary Classification
Harshit Varma*, Dheeraj Mysore Nagaraj, Karthikeyan Shanmugam

Interaction Asymmetry: A General Principle for Learning Composable Abstractions
Jack Brady, Julius von Kügelgen, Sébastien Lachapelle, Simon Buchholz, Thomas Kipf, Wieland Brendel

JetFormer: An Autoregressive Generative Model of Raw Images and Text
Michael Tschannen, André Susano Pinto, Alexander Kolesnikov

Large Language Models Are Interpretable Learners
Ruochen Wang*, Si Si, Felix Yu, Dorothea Wiesmann, Cho-Jui Hsieh, Inderjit Dhillon

Learning to Clarify: Multi-Turn Conversations With Action-Based Contrastive Self-Training
Maximillian Chen*, Ruoxi Sun, Tomas Pfister, Sercan Ö. Arik

Linear Transformer Topological Masking With Graph Random Features
Isaac Reid*, Avinava Dubey, Deepali Jain, Will Whitney, Amr Ahmed, Joshua Ainslie, Alex Bewley, Mithun George Jacob, Aranyak Mehta, David Rendleman, Connor Schenck, Richard E. Turner, René Wagner, Adrian Weller, Krzysztof Marcin Choromanski

LiNeS: Post-Training Layer Scaling Prevents Forgetting and Enhances Model Merging
Ke Wang, Nikolaos Dimitriadis, Alessandro Favero, Guillermo Ortiz-Jimenez, François Fleuret, Pascal Frossard

Measuring Memorization in RLHF for Code Completion
Aneesh Pappu, Billy Porter, Ilia Shumailov, Jamie Hayes

More Experts Than Galaxies: Conditionally-Overlapping Experts With Biologically-Inspired Fixed Routing
Sagi Shaier, Francisco Pereira, Katharina von der Wense, Lawrence E. Hunter, Matt Jones

Mufu: Multilingual Fused Learning for Low-Resource Translation with LLM
Zheng Wei Lim*, Nitish Gupta, Honglin Yu, Trevor Cohn

NL-Eye: Abductive NLI For Images
Mor Ventura, Michael Toker, Nitay Calderon, Zorik Gekhman, Yonatan Bitton, Roi Reichart

On the Transfer of Object-Centric Representation Learning
Aniket Rajiv Didolkar, Andrii Zadaianchuk, Anirudh Goyal, Michael C. Mozer, Yoshua Bengio, Georg Martius, Maximilian Seitzer

Personalized Representation From Personalized Generation
Shobhita Sundaram*, Julia Chae, Yonglong Tian*, Sara Beery, Phillip Isola

Re-Evaluating Open-Ended Evaluation of Large Language Models
Siqi Liu, Ian Gemp, Luke Marris, Georgios Piliouras, Nicolas Heess, Marc Lanctot

Relaxed Recursive Transformers: Effective Parameter Sharing With Layer-wise LoRA
Sangmin Bae*, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji, Seungyeon Kim, Tal Schuster

RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation
Konstantinos Bousmalis, Giulia Vezzani, Dushyant Rao, Coline Devin, Alex X. Lee, Maria Bauza, Todor Davchev, Yuxiang Zhou, Agrim Gupta*, Akhil Raju, Antoine Laurens, Claudio Fantacci, Valentin Dalibard, Martina Zambelli, Murilo F. Martins, Rugile Pevceviciute, Michiel Blokzijl, Misha Denil, Nathan Batchelor, Thomas Lampe, Emilio Parisotto, Konrad Żołna, Scott Reed, Sergio Gómez Colmenarejo, Jon Scholz, Abbas Abdolmaleki, Oliver Groth, Jean-Baptiste Regli, Oleg Sushkov, Tom Rothörl, José Enrique Chen, Yusuf Aytar, Dave Barker, Joy Ortiz, Martin Riedmiller, Jost Tobias Springenberg, Raia Hadsell, Francesco Nori, Nicolas Heess

Round and Round We Go! What Makes Rotary Positional Encodings Useful?
Federico Barbero*, Alex Vitvitskyi, Christos Perivolaropoulos, Razvan Pascanu, Petar Veličković

Scaling Laws for Downstream Task Performance in Machine Translation
Berivan Isik, Natalia Ponomareva, Hussein Hazimeh*, Dimitris Paparas, Sergei Vassilvitskii, Sanmi Koyejo*

SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pre-training of LLMs
Mohammad Mozaffari, Amir Yazdanbakhsh, Zhao Zhang, Maryam Mehri Dehnavi

Studying the Interplay Between the Actor and Critic Representations in Reinforcement Learning
Samuel Garcin, Trevor McInroe, Pablo Samuel Castro, Prakash Panangaden, Christopher G. Lucas, David Abel, Stefano V. Albrecht

To Clip or Not to Clip: The Dynamics of SGD With Gradient Clipping in High Dimensions
Noah Marshall, Ke Liang Xiao, Elliot Paquette, Atish Agarwala

Understanding and Mitigating Bottlenecks of State Space Models Through the Lens of Recency and Over-Smoothing
Peihao Wang, Ruisi Cai, Yuehao Wang, Jiajun Zhu, Pragya Srivastava, Zhangyang Wang, Pan Li

Video-STaR: Self-Training Enables Video Instruction Tuning With Any Supervision
Orr Zohar, Xiaohan Wang, Yonatan Bitton, Idan Szpektor, Serena Yeung-Levy

What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models
Ahmed Imtiaz Humayun, Ibtihel Amara, Cristina Vasconcelos, Deepak Ramachandran, Candice Schumann, Junfeng He, Katherine Heller, Golnoosh Farnadi, Negar Rostamzadeh, Mohammad Havaei

Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection
Yun Zhu, Jia-Chen Gu, Caitlin Sikora, Ho Ko, Yinxiao Liu, Chu-Cheng Lin, Lei Shu, Liangchen Luo, Lei Meng, Bang Liu, Jindong Chen

Adversarial Score Identity Distillation: Rapidly Surpassing the Teacher in One Step
Mingyuan Zhou, Huangjie Zheng, Yi Gu, Zhendong Wang, Hai Huang*

Beyond Worst-Case Dimensionality Reduction for Sparse Vectors
Sandeep Silwal, David P. Woodruff, Qiuyi Zhang

BOND: Aligning LLMs With Best-of-N Distillation
Pier Giuseppe Sessa, Robert Dadashi, Léonard Hussenot, Johan Ferret, Nino Vieillard, Alexandre Ramé, Bobak Shahriari, Sarah Perrin, Abram L. Friesen, Geoffrey Cideron, Sertan Girgin, Piotr Stanczyk, Andrea Michi, Danila Sinopalnikov, Sabela Ramos, Amélie Héliou, Aliaksei Severyn, Matt Hoffman, Nikola Momchev, Olivier Bachem

DiSK: Differentially Private Optimizer With Simplified Kalman Filter for Noise Reduction
Xinwei Zhang, Zhiqi Bu, Borja Balle, Mingyi Hong, Meisam Razaviyayn, Vahab Mirrokni

From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation
Xingchen Wan, Han Zhou, Ruoxi Sun, Sercan Ö. Arik

Grounding by Trying: LLMs With Reinforcement Learning-Enhanced Retrieval
Sheryl Hsu, Omar Khattab, Chelsea Finn, Archit Sharma

Inference-Aware Fine-Tuning for Best-of-N Sampling in Large Language Models
Yinlam Chow, Guy Tennenholtz, Izzeddin Gur, Vincent Zhuang, Bo Dai, Sridhar Thiagarajan, Craig Boutilier, Rishabh Agarwal, Aviral Kumar, Aleksandra Faust

InvestESG: A Multi-Agent Reinforcement Learning Benchmark for Studying Climate Investment as a Social Dilemma
Xiaoxuan Hou, Jiayi Yuan, Joel Z. Leibo, Natasha Jaques

Learn-by-Interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments
Hongjin Su, Ruoxi Sun, Jinsung Yoon, Pengcheng Yin, Tao Yu, Sercan Ö. Arik

Persistent Pre-training Poisoning of LLMs
Yiming Zhang, Javier Rando, Ivan Evtimov, Jianfeng Chi, Eric Michael Smith, Nicholas Carlini, Florian Tramèr, Daphne Ippolito

Privately Counting Partially Ordered Data
Matthew Joseph, Mónica Ribero, Alexander Yu

Reasoning With Latent Thoughts: On the Power of Looped Transformers
Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, Sashank J. Reddi

Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon
USVSN Sai Prashanth, Alvin Deng, Kyle O'Brien, Jyothir S V, Mohammad Aflah Khan, Jaydeep Borkar, Christopher A. Choquette-Choo, Jacob Ray Fuehne, Stella Biderman, Tracy Ke, Katherine Lee, Naomi Saphra

Revealing the 3D Cosmic Web Through Gravitationally Constrained Neural Fields
Brandon Zhao, Aviad Levis, Liam Connor, Pratul P. Srinivasan, Katherine L. Bouman

Scaling Wearable Foundation Models (see blog post)
Girish Narayanswamy*, Xin Liu, Kumar Ayush, Yuzhe Yang, Xuhai Xu, Shun Liao, Jake Garrison, Shyam A. Tailor, Jake Sunshine, Yun Liu, Tim Althoff, Shrikanth Narayanan, Pushmeet Kohli, Jiening Zhan, Mark Malhotra, Shwetak Patel, Samy Abdel-Ghaffar, Daniel McDuff

Semantic Image Inversion and Editing Using Rectified Stochastic Differential Equations
Litu Rout*, Yujia Chen, Nataniel Ruiz, Constantine Caramanis, Sanjay Shakkottai, Wen-Sheng Chu

The Journey Matters: Average Parameter Count Over Pre-training Unifies Sparse and Dense Scaling Laws
Tian Jin, Ahmed Imtiaz Humayun, Utku Evci, Suvinay Subramanian, Amir Yazdanbakhsh, Dan Alistarh, Gintare Karolina Dziugaite

Toward Understanding In-Context vs. In-Weight Learning
Bryan Chan, Xinyi Chen, András György, Dale Schuurmans

Unbounded: A Generative Infinite Game of Character Life Simulation
Jialu Li*, Yuanzhen Li, Neal Wadhwa, Yael Pritch, David E. Jacobs, Michael Rubinstein, Mohit Bansal, Nataniel Ruiz

Understanding and Enhancing Safety Mechanisms of LLMs via Safety-Specific Neuron
Yiran Zhao, Wenxuan Zhang, Yuxi Xie, Anirudh Goyal, Kenji Kawaguchi, Michael Shieh