mlsanityData to 7 Oct 2026

Generating Sequences by Learning to Self-Correct

arXiv 2211.00053 ↗First posted 31 Oct 2022cs.CLcited by 271

Not on the map. Why?

Cited by 103 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Sequence generation applications require satisfying semantic constraints, such as ensuring that programs are correct, using certain keywords, or avoiding undesirable content. Language models, whether fine-tuned or prompted with few-shot demonstrations, frequently violate these constraints, and lack a mechanism to iteratively revise their outputs. Moreover, some powerful language models are of extreme scale or inaccessible, making it inefficient, if not infeasible, to update their parameters for task-specific adaptation. We present Self-Correction, an approach that decouples an imperfect base generator (an off-the-shelf language model or supervised sequence-to-sequence model) from a separate corrector that learns to iteratively correct imperfect generations. To train the corrector, we propose an online training procedure that can use either scalar or natural language feedback on intermediate imperfect generations. We show that Self-Correction improves upon the base generator in three diverse generation tasks - mathematical program synthesis, lexically-constrained generation, and toxicity control - even when the corrector is much smaller than the base generator.

Key facts. Cited by 271 papers in the map (data through 7 Oct 2026). Builds on 41 of its 56 references that are in the map. First posted on arXiv 31 Oct 2022.

Builds on

41 of 56 references.

References · 41

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2020Learning to summarize from human feedbackcited by 2,562
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
29 more
  1. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  2. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  3. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  4. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  5. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  6. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  7. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  8. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 377
  9. 2022Self-critiquing models for assisting human evaluatorscited by 372
  10. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  11. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 252
  12. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  13. 2021Generated Knowledge Prompting for Commonsense Reasoningcited by 213
  14. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  15. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 159
  16. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 153
  17. 2020NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraintscited by 146
  18. 2018CGMH: Constrained Sentence Generation by Metropolis-Hastings Samplingcited by 93
  19. 2020Graph-based, Self-Supervised Program Repair from Diagnostic Feedbackcited by 93
  20. 2022PEER: A Collaborative Language Modelcited by 86
  21. 2021Break-It-Fix-It: Unsupervised Learning for Program Repaircited by 71
  22. 2019Corpora Generation for Grammatical Error Correctioncited by 53
  23. 2022Training Language Models with Language Feedbackcited by 48
  24. 2020Synthesize, Execute and Debug: Learning to Repair for Neural Program Synthesiscited by 39
  25. 2019Unsupervised Paraphrasing by Simulated Annealingcited by 37
  26. 2020Text Editing by Commandcited by 33
  27. 2021LM-Critic: Language Models for Unsupervised Grammatical Error Correctioncited by 33
  28. 2022Learning to Model Editing Processescited by 30
  29. 2020Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translationcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 57; 2024: 90; 2025: 86; 2026: 33.

271 papers.

Sort:

By area:

  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  4. 2023Teaching Large Language Models to Self-Debugcited by 963
  5. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  6. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  7. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  8. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  9. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
  10. 2023Language Models can Solve Computer Taskscited by 371
  11. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
  12. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 257
38 more
  1. 2025Training Language Models to Reason Efficientlycited by 242
  2. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  3. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
  4. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  5. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  6. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  7. 2024ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Modelscited by 200
  8. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 189
  9. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 162
  10. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  11. 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Taskscited by 137
  12. 2024From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Modelscited by 136
  13. 2023Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challengescited by 131
  14. 2022A Survey of Deep Learning for Mathematical Reasoningcited by 124
  15. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  16. 2023AutoMix: Automatically Mixing Language Modelscited by 116
  17. 2024Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffoldingcited by 115
  18. 2024Improving Multi-Agent Debate with Sparse Communication Topologycited by 103
  19. 2024GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinementscited by 101
  20. 2023ProAgent: Building Proactive Cooperative Agents with Large Language Modelscited by 97
  21. 2023Shepherd: A Critic for Language Model Generationcited by 96
  22. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revisioncited by 96
  23. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 90
  24. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  25. 2024CriticBench: Benchmarking LLMs for Critique-Correct Reasoningcited by 88
  26. 2023LLMs cannot find reasoning errors, but can correct them given the error locationcited by 85
  27. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 85
  28. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80
  29. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  30. 2023Training Socially Aligned Language Models on Simulated Social Interactionscited by 77
  31. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 74
  32. 2025Efficient Test-Time Scaling via Self-Calibrationcited by 74
  33. 2022Teaching Structured Vision&Language Concepts to Vision&Language Modelscited by 71
  34. 2024Self-Contrast: Better Reflection Through Inconsistent Solving Perspectivescited by 71
  35. 2025Self-rewarding correction for mathematical reasoningcited by 67
  36. 2024Generating CAD Code with Vision-Language Models for 3D Designscited by 61
  37. 2025Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitatecited by 61
  38. 2023CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modulescited by 58

50 most cited of 271 All 271 (JSON)