mlsanityData to 14 Oct 2026

Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning

arXiv 2402.10110 ↗First posted 15 Feb 2024cs.CL cs.AI cs.LGcited by 57

Not on the map. Why?

Abstract (arXiv)

Instruction tuning is critical to large language models (LLMs) for achieving better instruction following and task adaptation capabilities but its success heavily relies on the training data quality. Many recent methods focus on improving the data quality but often overlook the compatibility of the data with the student model being finetuned. This paper introduces Selective Reflection-Tuning, a novel paradigm that synergizes a teacher LLM's reflection and introspection for improving existing data quality with the data selection capability of the student LLM, to automatically refine existing instruction-tuning data. This teacher-student collaboration produces high-quality and student-compatible instruction-response pairs, resulting in sample-efficient instruction tuning and LLMs of superior performance. Selective Reflection-Tuning is a data augmentation and synthesis that generally improves LLM finetuning and self-improvement without collecting brand-new data. We apply our method to Alpaca and WizardLM data and achieve much stronger and top-tier 7B and 13B LLMs.

Key facts. Cited by 57 papers in the map (data through 14 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

43 of 58 references.

References · 43

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  12. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
31 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  5. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  6. 2023LIMA: Less Is More for Alignmentcited by 1,100
  7. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  8. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,031
  9. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  10. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 839
  11. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 748
  12. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  13. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  14. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  15. 2022Large Language Models Can Self-Improvecited by 664
  16. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 625
  17. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
  18. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  19. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
  20. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
  21. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 252
  22. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  23. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  24. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuningcited by 150
  25. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  26. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 138
  27. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  28. 2023Shepherd: A Critic for Language Model Generationcited by 97
  29. 2020Look at the First Sentence: Position Bias in Question Answeringcited by 84
  30. 2023Rethinking the Instruction Quality: LIFT is What You Needcited by 34
  31. 2023Koala: An Index for Quantifying Overlaps with Pre-training Corporacited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 26; 2025: 24; 2026: 7.

57 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  2. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 125
  3. 2024A Survey on Self-Evolution of Large Language Modelscited by 91
  4. 2025DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMscited by 84
  5. 2025Self-rewarding correction for mathematical reasoningcited by 70
  6. 2025The Best Instruction-Tuning Data are Those That Fitcited by 52
  7. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 51
  8. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  9. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 41
  10. 2024SuperCorrect: Advancing Small LLM Reasoning with Thought Template Distillation and Self-Correctioncited by 40
  11. 2024Agent Design Pattern Catalogue: A Collection of Architectural Patterns for Foundation Model based Agentscited by 35
  12. 2024Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statementscited by 34
38 more
  1. 2024A Survey of Small Language Modelscited by 31
  2. 2025ATLaS: Agent Tuning via Learning Critical Stepscited by 31
  3. 2026Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoningcited by 23
  4. 2025LEMMA: Learning from Errors for MatheMatical Advancement in LLMscited by 21
  5. 2025Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generationcited by 21
  6. 2025Towards Visual Text Grounding of Multimodal Large Language Modelcited by 18
  7. 2024Stronger Models are NOT Stronger Teachers for Instruction Tuningcited by 16
  8. 2024Not All Samples Should Be Utilized Equally: Towards Understanding and Improving Dataset Distillationcited by 14
  9. 2024Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuningcited by 14
  10. 2024Distilling Instruction-following Abilities of Large Language Models with Task-aware Curriculum Planningcited by 13
  11. 2026Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignmentcited by 13
  12. 2025Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencodercited by 12
  13. 2025From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMscited by 12
  14. 2024RuleR: Improving LLM Controllability by Rule-based Data Recyclingcited by 11
  15. 2024Prefix Text as a Yarn: Eliciting Non-English Alignment in Foundation Language Modelcited by 10
  16. 2024Federated Data-Efficient Instruction Tuning for Large Language Modelscited by 10
  17. 2025DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Modelscited by 10
  18. 2024Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuningcited by 9
  19. 2024Only-IF:Revealing the Decisive Effect of Instruction Diversity on Generalizationcited by 8
  20. 2024Semi-supervised Fine-tuning for Large Language Modelscited by 8
  21. 2025Aligning Instruction Tuning with Pre-trainingcited by 7
  22. 2025ALinFiK: Learning to Approximate Linearized Future Influence Kernel for Scalable Third-Party LLM Data Valuationcited by 7
  23. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7
  24. 2024IterSelectTune: An Iterative Training Framework for Efficient Instruction-Tuning Data Selectioncited by 6
  25. 2024Retrospective Learning from Interactionscited by 6
  26. 2025A Survey on Proactive Defense Strategies Against Misinformation in Large Language Modelscited by 6
  27. 2024Preference-Oriented Supervised Fine-Tuning: Favoring Target Model Over Aligned Large Language Modelscited by 5
  28. 2025On Accelerating Edge AI: Optimizing Resource-Constrained Environmentscited by 5
  29. 2025MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretrainingcited by 5
  30. 2024Multi-Objective Linguistic Control of Large Language Modelscited by 4
  31. 2025MergeIT: From Selection to Merging for Efficient Instruction Tuningcited by 3
  32. 2025RICo: Refined In-Context Contribution for Automatic Instruction-Tuning Data Selectioncited by 3
  33. 2024NILE: Internal Consistency Alignment in Large Language Modelscited by 2
  34. 2024Large Language Models as Realistic Microservice Trace Generatorscited by 2
  35. 2025DFQ-ViT: Data-Free Quantization for Vision Transformers without Fine-tuningcited by 2
  36. 2024From Symbolic Tasks to Code Generation: Diversification Yields Better Task Performerscited by 1
  37. 2024Boosting LLM via Learning from Data Iteratively and Selectivelycited by 1
  38. 2025LLMSynthor: Macro-Aligned Micro-Records Synthesis with Large Language Modelscited by 1

50 most cited of 57 All 57 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning
  2. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models
  3. 2024ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning
  4. 2023Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
  5. 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
  6. 2023Tuna: Instruction Tuning using Feedback from Large Language Models
  7. 2024Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning
  8. 2025LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning