mlsanityData to 14 Oct 2026

Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning

arXiv 2310.11716 ↗First posted 18 Oct 2023cs.CLcited by 26

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Recent advancements in Large Language Models (LLMs) have expanded the horizons of natural language understanding and generation. Notably, the output control and alignment with the input of LLMs can be refined through instruction tuning. However, as highlighted in several studies, low-quality data in the training set are usually detrimental to instruction tuning, resulting in inconsistent or even misleading LLM outputs. We propose a novel method, termed "reflection-tuning," which addresses the problem by self-improvement and judging capabilities of LLMs. This approach utilizes an oracle LLM to recycle the original training data by introspecting and enhancing the quality of instructions and responses in the data. Extensive experiments on widely used evaluation benchmarks show that LLMs trained with our recycled data outperform those trained with existing datasets in various benchmarks.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 28 of its 43 references that are in the map. First posted on arXiv 18 Oct 2023.

Builds on

28 of 43 references.

References · 28

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  11. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  12. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
16 more
  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  4. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  5. 2023Teaching Large Language Models to Self-Debugcited by 988
  6. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  7. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  8. 2023Instruction Tuning with GPT-4cited by 686
  9. 2022Large Language Models Can Self-Improvecited by 664
  10. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  11. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  12. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  13. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  14. 2019Show Your Work: Improved Reporting of Experimental Resultscited by 169
  15. 2023Self-Alignment with Instruction Backtranslationcited by 151
  16. 2020Look at the First Sentence: Position Bias in Question Answeringcited by 84

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 17; 2025: 7; 2026: 2.

26 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 138
  3. 2024Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuningcited by 85
  4. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 41
  5. 2024SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Modelscited by 39
  6. 2024Break the Chain: Large Language Models Can be Shortcut Reasonerscited by 32
  7. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  8. 2024A Survey of Small Language Modelscited by 31
  9. 2025Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Modelscited by 20
  10. 2024CodeACT: Code Adaptive Compute-efficient Tuning Framework for Code LLMscited by 15
  11. 2024Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuningcited by 14
  12. 2024Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planningcited by 13
14 more
  1. 2025Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencodercited by 12
  2. 2025ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflectioncited by 12
  3. 2024FANNO: Augmenting High-Quality Instruction Data with Open-Sourced LLMs Onlycited by 11
  4. 2024Multi-Programming Language Sandbox for LLMscited by 10
  5. 2024Honesty to Subterfuge: In-Context Reinforcement Learning Can Make Honest Models Reward Hackcited by 8
  6. 2024Privacy-preserved LLM Cascade via CoT-enhanced Policy Learningcited by 8
  7. 2025DISCO Balances the Scales: Adaptive Domain- and Difficulty-Aware Reinforcement Learning on Imbalanced Datacited by 7
  8. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Modelscited by 5
  9. 2024Whispers in the Machine: Confidentiality in Agentic Systemscited by 2
  10. 2024DELIA: Diversity-Enhanced Learning for Instruction Adaptation in Large Language Modelscited by 0
  11. 2024Enhancing Security Control Production With Generative AIcited by 0
  12. 2025Error Notebook-Guided, Training-Free Part Retrieval in 3D CAD Assemblies via Vision-Language Modelscited by 0
  13. 2026Reflective Translation: Improving Low-Resource Machine Translation via Structured Self-Reflectioncited by 0
  14. 2026DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning
  2. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models
  3. 2025Aligning Instruction Tuning with Pre-training
  4. 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
  5. 2023Rethinking the Instruction Quality: LIFT is What You Need
  6. 2023Tuna: Instruction Tuning using Feedback from Large Language Models
  7. 2024RE-Adapt: Reverse Engineered Adaptation of Large Language Models
  8. 2025Instruction Tuning with and without Context: Behavioral Shifts and Downstream Impact