mlsanityData to 14 Oct 2026

Large Language Models Can Self-Improve

arXiv 2210.11610 ↗First posted 20 Oct 2022cs.CLcited by 664

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) have achieved excellent performances in various tasks. However, fine-tuning an LLM requires extensive supervision. Human, on the other hand, may improve their reasoning abilities by self-thinking without external inputs. In this work, we demonstrate that an LLM is also capable of self-improving with only unlabeled datasets. We use a pre-trained LLM to generate "high-confidence" rationale-augmented answers for unlabeled questions using Chain-of-Thought prompting and self-consistency, and fine-tune the LLM using those self-generated solutions as target outputs. We show that our approach improves the general reasoning ability of a 540B-parameter LLM (74.4%->82.1% on GSM8K, 78.2%->83.0% on DROP, 90.0%->94.4% on OpenBookQA, and 63.4%->67.9% on ANLI-A3) and achieves state-of-the-art-level performance, without any ground truth label. We conduct ablation studies and show that fine-tuning on reasoning is critical for self-improvement.

Key facts. Cited by 664 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 47 of its 65 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

47 of 65 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Language Models (Mostly) Know What They Know11 Jul

Other references · 46

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2017On Calibration of Modern Neural Networkscited by 4,960
  9. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  12. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
34 more
  1. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  2. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  3. 2022Emergent Abilities of Large Language Modelscited by 2,512
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  5. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  6. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  8. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  9. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  10. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  11. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  12. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  13. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  14. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 807
  15. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  16. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  17. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  18. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  19. 2013Do Deep Nets Really Need to be Deep?cited by 474
  20. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  21. 2021Semi-Supervised Semantic Segmentation with Cross Pseudo Supervisioncited by 463
  22. 2022Can language models learn from explanations in context?cited by 249
  23. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  24. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 187
  25. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  26. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  27. 2020WT5?! Training Text-to-Text Models to Explain their Predictionscited by 148
  28. 2022Learning by Distilling Contextcited by 138
  29. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  30. 2020Evaluating Explanations: How much do explanations from the teacher aid students?cited by 94
  31. 2019Automatic adaptation of object detectors to new domains using self-trainingcited by 66
  32. 2015Bayesian Dark Knowledgecited by 45
  33. 2022Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Modelcited by 20
  34. 2022OPERA:Operation-Pivoted Discrete Reasoning over Textcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 14; 2023: 150; 2024: 208; 2025: 190; 2026: 102.

664 papers.

Sort:

By area:

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  4. 2023A Survey of Large Language Modelscited by 2,576
  5. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  6. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  7. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  8. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  9. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  10. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  11. 2025Kimi K2: Open Agentic Intelligencecited by 567
  12. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
38 more
  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  2. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
  3. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  4. 2023Cognitive Architectures for Language Agentscited by 429
  5. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  6. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
  7. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  8. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 356
  9. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  10. 2022Large Language Models Are Reasoning Teacherscited by 329
  11. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  12. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  13. 2022Teaching Small Language Models to Reasoncited by 284
  14. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  15. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 282
  16. 2025TTRL: Test-Time Reinforcement Learningcited by 275
  17. 2024TextGrad: Automatic "Differentiation" via Textcited by 273
  18. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  19. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
  20. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  21. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  22. 2023FireAct: Toward Language Agent Fine-tuningcited by 256
  23. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  24. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 251
  25. 2023Self-Consuming Generative Models Go MADcited by 250
  26. 2024On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surveycited by 244
  27. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  28. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  29. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  30. 2024Chain-of-Thought Reasoning Without Promptingcited by 210
  31. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 205
  32. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 196
  33. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
  34. 2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 192
  35. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  36. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 179
  37. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  38. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 166

50 most cited of 664 All 664 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Large Language Models Can Self-Improve in Long-context Reasoning
  2. 2025The Self-Improvement Paradox: Can Language Models Bootstrap Reasoning Capabilities without External Scaffolding?
  3. 2025Leveraging Reasoning Model Answers to Enhance Non-Reasoning Model Capability
  4. 2025Think, Prune, Train, Improve: Scaling Reasoning without Scaling Models
  5. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  6. 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
  7. 2024Self-Training Meets Consistency: Improving LLMs' Reasoning with Consistency-Driven Rationale Evaluation
  8. 2025Self-Training Large Language Models with Confident Reasoning