Latent Reasoning with Supervised Thinking States

Chain-of-Thought (CoT) improves LLM reasoning but amplifies decoding latency and memory usage. While latent reasoning attempts to shift this computation to internal hidden states, it is hindered by the parallel nature of the Transformer prefill—which blocks sequential deep-to-shallow information flow—and the high training costs of recurrent optimization. We propose a supervised latent reasoning approach that enables efficient sequential computation during prefill. Our method augments an LLM with a recurrent pathway to propagate intermediate "thinking" tokens from deep to shallow layers without expanding the KV-cache. Unlike prior unsupervised methods, we utilize structured supervision to train this mechanism via teacher forcing, eliminating the need for Backpropagation Through Time (BPTT). Evaluations on state-tracking and reasoning benchmarks demonstrate that our approach outperforms existing latent baselines and approaches CoT performance, effectively combining the reasoning power of CoT with the inference efficiency of standard models.