mlsanityData to 7 Oct 2026

Self-Refine: Iterative Refinement with Self-Feedback

arXiv 2303.17651 ↗First posted 30 Mar 2023cs.CL cs.AI cs.LGcited by 3,317

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Like humans, large language models (LLMs) do not always generate the best output on their first try. Motivated by how humans refine their written text, we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement. The main idea is to generate an initial output using an LLMs; then, the same LLMs provides feedback for its output and uses it to refine itself, iteratively. Self-Refine does not require any supervised training data, additional training, or reinforcement learning, and instead uses a single LLM as the generator, refiner, and feedback provider. We evaluate Self-Refine across 7 diverse tasks, ranging from dialog response generation to mathematical reasoning, using state-of-the-art (GPT-3.5, ChatGPT, and GPT-4) LLMs. Across all evaluated tasks, outputs generated with Self-Refine are preferred by humans and automatic metrics over those generated with the same LLM using conventional one-step generation, improving by ~20% absolute on average in task performance. Our work demonstrates that even state-of-the-art LLMs like GPT-4 can be further improved at test time using our simple, standalone approach.

Key facts. Cited by 3,317 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 36 of its 52 references that are in the map. First posted on arXiv 30 Mar 2023.

Builds on

36 of 52 references.

References · 36

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  10. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  11. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  12. 2022PAL: Program-aided Language Modelscited by 635
24 more
  1. 2022Self-critiquing models for assisting human evaluatorscited by 372
  2. 2023GPTScore: Evaluate as You Desirecited by 357
  3. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  4. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  5. 2022Generating Sequences by Learning to Self-Correctcited by 271
  6. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  7. 2021CodeNet: A Large-Scale AI for Code Dataset for Learning a Diversity of Coding Taskscited by 199
  8. 2022Re3: Generating Longer Stories With Recursive Reprompting and Revisioncited by 193
  9. 2018Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfercited by 191
  10. 2020Unsupervised Evaluation of Interactive Dialog with DialoGPTcited by 139
  11. 2018Style Transfer Through Back-Translationcited by 124
  12. 2023Learning Performance-Improving Code Editscited by 103
  13. 2020Graph-based, Self-Supervised Program Repair from Diagnostic Feedbackcited by 93
  14. 2022PEER: A Collaborative Language Modelcited by 86
  15. 2022Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revisioncited by 46
  16. 2022Rainier: Reinforced Knowledge Introspector for Commonsense Question Answeringcited by 43
  17. 2021Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedbackcited by 35
  18. 2021NL-EDIT: Correcting semantic parse errors through natural language interactioncited by 33
  19. 2022Learning to Model Editing Processescited by 30
  20. 2021Think about it! Improving defeasible reasoning by first modeling the question scenariocited by 20
  21. 2018Guide Me: Interacting with Deep Networkscited by 16
  22. 2019Improving Natural Language Interaction with Robots Using Advicecited by 9
  23. 2021Interscript: A dataset for interactive learning of scripts through error feedbackcited by 9
  24. 2022Triangulating Python Performance Issues with Scalenecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 295; 2024: 633; 2025: 957; 2026: 1432.

3,317 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  5. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  7. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  9. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  10. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  11. 2023Teaching Large Language Models to Self-Debugcited by 963
  12. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
38 more
  1. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  2. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  3. 2023Large Language Models as Optimizerscited by 830
  4. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  5. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  6. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  7. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  8. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  9. 2023Instruction Tuning with GPT-4cited by 676
  10. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  11. 2024A Survey on Large Language Models for Code Generationcited by 646
  12. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  13. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  14. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  15. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  16. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  17. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  18. 2023Cognitive Architectures for Language Agentscited by 422
  19. 2024Understanding the planning of LLM agents: A surveycited by 414
  20. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  21. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  22. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  23. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
  24. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  25. 2024Automated Design of Agentic Systemscited by 360
  26. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  27. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  28. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
  29. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 319
  30. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  31. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  32. 2023Tool Learning with Foundation Modelscited by 302
  33. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 297
  34. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  35. 2023Large Language Models as Tool Makerscited by 293
  36. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
  37. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  38. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292

50 most cited of 3,317 All 3,317 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning
  2. 2024Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement
  3. 2025Self-Correcting Large Language Models: Generation vs. Multiple Choice
  4. 2025Self-Evolving Critique Abilities in Large Language Models
  5. 2025Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs
  6. 2024Self-Refine Instruction-Tuning for Aligning Reasoning in Language Models
  7. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlook
  8. 2024Teaching LLMs to Refine with Tools