mlsanityData to 14 Oct 2026

ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection

arXiv 2505.16475 ↗First posted 22 May 2025cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

We present a novel pipeline, ReflectEvo, to demonstrate that small language models (SLMs) can enhance meta introspection through reflection learning. This process iteratively generates self-reflection for self-training, fostering a continuous and self-evolving process. Leveraging this pipeline, we construct ReflectEvo-460k, a large-scale, comprehensive, self-generated reflection dataset with broadened instructions and diverse multi-domain tasks. Building upon this dataset, we demonstrate the effectiveness of reflection learning to improve SLMs' reasoning abilities using SFT and DPO with remarkable performance, substantially boosting Llama-3 from 52.4% to 71.2% and Mistral from 44.4% to 71.1%. It validates that ReflectEvo can rival or even surpass the reasoning capability of the three prominent open-sourced models on BIG-bench without distillation from superior models or fine-grained human annotation. We further conduct a deeper analysis of the high quality of self-generated reflections and their impact on error localization and correction. Our work highlights the potential of continuously enhancing the reasoning performance of SLMs through iterative reflection learning in the long run.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

36 of 49 references.

References · 36

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  8. 2022Large Language Models Can Self-Improvecited by 664
  9. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  10. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  11. 2024NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Modelscited by 472
  12. 2023Statistical Rejection Sampling Improves Preference Optimizationcited by 323
24 more
  1. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
  2. 2022Generating Sequences by Learning to Self-Correctcited by 277
  3. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  4. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 202
  5. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 197
  6. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 165
  7. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimizationcited by 112
  8. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoningcited by 86
  9. 2024MALT: Improving Reasoning with Multi-Agent LLM Trainingcited by 76
  10. 2023Language Model Self-improvement by Reinforcement Learning Contemplationcited by 75
  11. 2024Vision-Language Models Can Self-Improve Reasoning via Reflectioncited by 69
  12. 2023Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasonerscited by 63
  13. 2024LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraintscited by 35
  14. 2024Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Taskscited by 32
  15. 2024Re-ReST: Reflection-Reinforced Self-Training for Language Agentscited by 29
  16. 2024I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigmcited by 27
  17. 2023Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuningcited by 26
  18. 2024Self-Training with Direct Preference Optimization Improves Chain-of-Thought Reasoningcited by 26
  19. 2024An Efficient Recipe for Long Context Extension via Middle-Focused Positional Encodingcited by 16
  20. 2024Reflect-RL: Two-Player Online RL Fine-Tuning for LMscited by 13
  21. 2024Mars: Situated Inductive Reasoning in an Open-World Environmentcited by 11
  22. 2024FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs' Responsiveness to Human Feedbackcited by 11
  23. 2024RAM: Towards an Ever-Improving Memory System by Learning from Communicationscited by 6
  24. 2024ExoViP: Step-by-step Verification and Exploration with Exoskeleton Modules for Compositional Visual Reasoningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2025Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMscited by 12
  2. 2025RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Samplingcited by 6
  3. 2026Retrospective Progress-Aware Self-Refinement for LLM Agent Trainingcited by 6
  4. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  5. 2026Tailoring the Curriculum: Student-Centered Reasoning Distillation via Dynamic Data-Model Compatibilitycited by 2
  6. 2025MetaCrit: A Critical Thinking Framework for Self-Regulated LLM Reasoningcited by 1
  7. 2026PACE-RAG: Patient-Aware Contextual and Evidence-Constrained RAG for Clinical Drug Recommendationcited by 1
  8. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
  9. 2026LM-Lexicon: Improving Definition Modeling via Harmonizing Semantic Expertscited by 0
  10. 2026SOLAR: A Self-Optimizing Open-Ended Autonomous Agent for Lifelong Learning and Continual Adaptationcited by 0
  11. 2026Reflective Recovery: A Self-Supervised Method for Reasoning by Learning from Mistakescited by 0
  12. 2026MENTOR: Learning from Failure with On-Policy Pedagogical Interventioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction
  2. 2025First Try Matters: Revisiting the Role of Reflection in Reasoning Models
  3. 2023Democratizing Reasoning Ability: Tailored Learning from Large Language Model
  4. 2026A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvement
  5. 2025Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning
  6. 2024When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Models
  7. 2024Self-Reflection Makes Large Language Models Safer, Less Biased, and Ideologically Neutral
  8. 2025From Emergence to Control: Probing and Modulating Self-Reflection in Language Models