mlsanityData to 14 Oct 2026

When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain

arXiv 2609.36535 ↗First posted 29 Sep 2026cs.CLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Self-evolution lets large language models (LLMs) improve iteratively using their own generated data, but often suffers from self-evolution degeneration: performance improves, plateaus, then declines. Existing methods address this issue at the component level, targeting either the Questioner or the Solver, and overlook that self-evolution is a tightly coupled system. We propose a holistic framework based on learnable information gain, which measures how much novel, parameterizable information a round provides relative to the previous round. Theoretically, this gain equals the Kullback-Leibler divergence between the two rounds' data distributions plus their entropy change. Practically, it is estimated by fitting a small language model to the previous round and scoring new data via negative log-likelihood. Based on this diagnostic, we propose ATRI (Adaptive Training Regulation via Information-gain), which reweights samples within a round and halts training across rounds when information gain remains low. Experiments on popular datasets demonstrate the superiority of our proposal.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 50 of its 59 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

50 of 59 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Large Language Models Can Self-Improve20 Oct
  2. 2022STaR: Bootstrapping Reasoning With Reasoning28 Mar

Other references · 48

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  5. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  6. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  7. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  10. 2024Self-Rewarding Language Modelscited by 681
  11. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  12. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
36 more
  1. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  2. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 485
  3. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 399
  4. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
  5. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  6. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  7. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  8. 2023Self-Consuming Generative Models Go MADcited by 250
  9. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 236
  10. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
  11. 2020Self-Distillation Amplifies Regularization in Hilbert Spacecited by 173
  12. 2024Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Datacited by 151
  13. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
  14. 2025BIG-Bench Extra Hardcited by 125
  15. 2024Self-Improvement in Language Models: The Sharpening Mechanismcited by 104
  16. 2023On the Stability of Iterative Retraining of Generative Models on their own Datacited by 100
  17. 2024Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Modelscited by 84
  18. 2024How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapsecited by 79
  19. 2018The Description Length of Deep Learning Modelscited by 72
  20. 2025Can Large Reasoning Models Self-Train?cited by 68
  21. 2024Strong Model Collapsecited by 65
  22. 2026From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligencecited by 41
  23. 2025Inference-Time Reward Hacking in Large Language Modelscited by 34
  24. 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Datacited by 34
  25. 2024Progress or Regress? Self-Improvement Reversal in Post-trainingcited by 23
  26. 2025Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Modelscited by 23
  27. 2026G-Zero: Self-Play for Open-Ended Generation from Zero Datacited by 17
  28. 2026R-Diverse: Mitigating Diversity Illusion in Self-Play LLM Trainingcited by 10
  29. 2026Scaling Self-Play with Self-Guidancecited by 10
  30. 2026The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societiescited by 7
  31. 2025Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gapcited by 6
  32. 2026When Self-Belief Misleads: Active Label Acquisition for Reinforcement Learning with Verifiable Rewardscited by 6
  33. 2025Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theorycited by 5
  34. 2026Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RLcited by 4
  35. 2026Self-Improvement Can Self-Regress: The Rise-and-Collapse Failure Mode of LLM Self-Trainingcited by 4
  36. 2026Benign Alone, Harmful Together: Exploiting Experience Composition in Self-Evolving LLM Agentscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Is Enough Enough in Self-Evolving LLM Systems?
  2. 2025Evolving LLMs' Self-Refinement Capability via Synergistic Training-Inference Optimization
  3. 2026Learning to Self-Evolve
  4. 2026On the Generalization Gap in Self-Evolving Language Model Reasoning
  5. 2025Self-Adapting Language Models
  6. 2024A Survey on Self-Evolution of Large Language Models
  7. 2025Self-Evolving Critique Abilities in Large Language Models
  8. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback