mlsanityData to 14 Oct 2026

Stateful Reasoning via Insight Replay

arXiv 2605.14457 ↗First posted 14 May 2026cs.AIcited by 0

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-Thought (CoT) reasoning has become a foundation for eliciting multi-step reasoning in large language models, but recent studies show that its benefits do not scale monotonically with chain length: while longer CoT generally enables a model to tackle harder problems, on a given problem, accuracy typically increases with CoT length up to a point, after which it declines. We identify a major cause of this phenomenon: as the CoT grows, the model's attention to critical insights produced earlier in the trace gradually weakens, making those insights progressively less accessible when they are most needed. Therefore, we propose InsightReplay, a stateful reasoning approach in which the model periodically extracts critical insights from its reasoning trace and replays them near the active generation frontier, keeping them accessible as the reasoning scales. Extensive experiments on a 2!×!3!×!4 benchmark grid, covering model scales 8B, 30B, model families Qwen3.5, DeepSeek-R1-Distill-Qwen, Gemma-4, and reasoning benchmarks AIME, HMMT, GPQA Diamond, LiveCodeBench v5, show that 3-round InsightReplay yields accuracy gains across all 24 settings, with an averaged improvement of +1.65 points over standard CoT, and a largest single-setting gain of +9.2 points on R1-Distill-32B's LiveCodeBench v5 subset. Our results suggest that the effectiveness of test-time scaling depends not only on how much a model reasons, but also on whether critical intermediate insights remain accessible throughout long reasoning trajectories.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 26 of its 32 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

26 of 32 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Thought Anchors: Which LLM Reasoning Steps Matter?23 Jun
  2. 2025PENCIL: Long Thoughts with Short Memory18 Mar
  3. 2025When More is Less: Understanding Chain-of-Thought Length in LLMs11 Feb
  4. 2022Large Language Models are Zero-Shot Reasoners24 May
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 20

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  7. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  10. 2025MathArena: Evaluating LLMs on Uncontaminated Math Competitionscited by 297
  11. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMscited by 249
  12. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectivenesscited by 119
8 more
  1. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 115
  2. 2024Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capabilitycited by 83
  3. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Modelscited by 54
  4. 2025Inverse Scaling in Test-Time Computecited by 50
  5. 2025Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropycited by 38
  6. 2026Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokenscited by 22
  7. 2025Rethinking Thinking Tokens: LLMs as Improvement Operatorscited by 21
  8. 2025Not All Tokens Are What You Need In Thinkingcited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoning
  2. 2025Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamics
  3. 2024Fine-Tuning on Diverse Reasoning Chains Drives Within-Inference CoT Refinement in LLMs
  4. 2026Does Out-of-Sight Equal Out-of-Mind in CoT Monitorability?
  5. 2026The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics
  6. 2025Early Stopping Chain-of-thoughts in Large Language Models
  7. 2025From Reasoning to Super-Intelligence: A Search-Theoretic Perspective
  8. 2025Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models