MaxIM: Maximally Informative Incremental Summarization via Reinforcement Learning

Jihwan Jeong
Guy Tennenholtz
Fortieth Annual Conference on Neural Information Processing Systems (NeurIPS-26), Sydney (2026)

Abstract

Continuous streaming applications require AI systems to maintain bounded episodic memories of unbounded interaction histories. While optimizing these memory updates for downstream task utility is the ultimate goal, standard reinforcement learning (RL) suffers from severe credit assignment bottlenecks due to sparse, delayed rewards. We introduce MaxIM (Maximally Informative Language Model), a framework that formulates incremental summarization as a capacity-constrained, information-theoretic reward shaping problem. Grounded in value suboptimality bounds, we mathematically prove that minimizing sequential information loss requires maximizing the predictive sufficiency of the summary. To prevent LLMs from hallucinating during this predictive optimization, we introduce a secondary recursive consistency constraint that enforces strict historical grounding. Because computing true mutual information is intractable, MaxIM leverages oracle context during offline training to learn variational lower bounds via a parameterized critic. This provides the RL policy with dense, step-by-step regularizers that solve the sparse credit assignment problem. Empirical evaluation on long-horizon benchmarks---including multi-turn user simulation and goal-directed scientific reading---demonstrates that MaxIM achieves significant improvements across comprehensive autorater metrics and downstream task utility compared to state-of-the-art baselines, all without requiring expensive human preference labels.

Research Areas

×