mlsanityData to 7 Oct 2026

LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement

arXiv 2403.15042 ↗First posted 22 Mar 2024cs.CLcited by 69

Not on the map. Why?

Abstract (arXiv)

Pretrained large language models (LLMs) are currently state-of-the-art for solving the vast majority of natural language processing tasks. While many real-world applications still require fine-tuning to reach satisfactory levels of performance, many of them are in the low-data regime, making fine-tuning challenging. To address this, we propose LLM2LLM, a targeted and iterative data augmentation strategy that uses a teacher LLM to enhance a small seed dataset by augmenting additional data that can be used for fine-tuning on a specific task. LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data, (2) evaluates and extracts data points that the model gets wrong, and (3) uses a teacher LLM to generate synthetic data based on these incorrect data points, which are then added back into the training data. This approach amplifies the signal from incorrectly predicted data points by the LLM during training and reintegrates them into the dataset to focus on more challenging examples for the LLM. Our results show that LLM2LLM significantly enhances the performance of LLMs in the low-data regime, outperforming both traditional fine-tuning and other data augmentation baselines. LLM2LLM reduces the dependence on labor-intensive data curation and paves the way for more scalable and performant LLM solutions, allowing us to tackle data-constrained domains and tasks. We achieve improvements up to 24.2% on the GSM8K dataset, 32.6% on CaseHOLD, 32.0% on SNIPS, 52.6% on TREC and 39.8% on SST-2 over regular fine-tuning in the low-data regime using a Llama-2-7B student model. Our code is available at https://github.com/SqueezeAILab/LLM2LLM .

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Builds on 56 of its 80 references that are in the map. First posted on arXiv 22 Mar 2024.

Builds on

56 of 80 references.

References · 56

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
38 more
  1. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  2. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  5. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  6. 2023LIMA: Less Is More for Alignmentcited by 1,080
  7. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  8. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
  9. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  10. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  11. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  12. 2024Self-Rewarding Language Modelscited by 654
  13. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  14. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  15. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  16. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  17. 2018Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfacescited by 376
  18. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  19. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  20. 2023Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicinecited by 319
  21. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  22. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  23. 2022Is GPT-3 a Good Data Annotator?cited by 208
  24. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 199
  25. 2021Want To Reduce Labeling Cost? GPT-3 Can Helpcited by 196
  26. 2021GPT3Mix: Leveraging Large-scale Language Models for Text Augmentationcited by 185
  27. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  28. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  29. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 157
  30. 2023Self-Alignment with Instruction Backtranslationcited by 149
  31. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 128
  32. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  33. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performancecited by 102
  34. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 93
  35. 2022Language Models Can Teach Themselves to Program Bettercited by 75
  36. 2023TinyGSM: achieving >80% on GSM8k with small language modelscited by 68
  37. 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Taskscited by 62
  38. 2023MM-LIMA: Less Is More for Alignment in Multi-Modal Datasetscited by 61

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 32; 2025: 23; 2026: 14.

69 papers.

Sort:

By area:

  1. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  2. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  3. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 94
  4. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 90
  5. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  6. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  7. 2024Instruction Pre-Training: Language Models are Supervised Multitask Learnerscited by 63
  8. 2024TinyAgent: Function Calling at the Edgecited by 55
  9. 2025SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoningcited by 41
  10. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 40
  11. 2024A Survey of Small Language Modelscited by 31
  12. 2024I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigmcited by 27
38 more
  1. 2024Efficient and Scalable Estimation of Tool Representations in Vector Spacecited by 22
  2. 2024Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Modelcited by 21
  3. 2024ProVision: Programmatically Scaling Vision-centric Instruction Data for Multimodal Language Modelscited by 21
  4. 2025LEMMA: Learning from Errors for MatheMatical Advancement in LLMscited by 20
  5. 2024Bootstrapping Language-Guided Navigation Learning with Self-Refining Data Flywheelcited by 18
  6. 2024ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Modelscited by 14
  7. 2024Interactive Evolution: A Neural-Symbolic Self-Training Framework For Large Language Modelscited by 13
  8. 2024COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesiscited by 13
  9. 2024On Domain-Adaptive Post-Training for Multimodal Large Language Modelscited by 13
  10. 2024Montessori-Instruct: Generate Influential Training Data Tailored for Student Learningcited by 12
  11. 2024LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvementcited by 11
  12. 2024Math Neurosurgery: Isolating Language Models' Math Reasoning Abilities Using Only Forward Passescited by 10
  13. 2025Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMscited by 9
  14. 2025A Survey on Efficient Large Language Model Training: From Data-centric Perspectivescited by 9
  15. 2024Honesty to Subterfuge: In-Context Reinforcement Learning Can Make Honest Models Reward Hackcited by 8
  16. 2024Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignmentcited by 7
  17. 2025Aligning Instruction Tuning with Pre-trainingcited by 7
  18. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7
  19. 2024EF-LLM: Energy Forecasting LLM with AI-assisted Automation, Enhanced Sparse Prediction, Hallucination Detectioncited by 6
  20. 2024Language Models as Continuous Self-Evolving Data Engineerscited by 6
  21. 2025Theoretical Modeling of Large Language Model Self-Improvement Training Dynamics Through Solver-Verifier Gapcited by 6
  22. 2025Towards Active Synthetic Data Generation for Finetuning Language Modelscited by 6
  23. 2024Uncertainty-Guided Self-Questioning and Answering for Video-Language Alignmentcited by 4
  24. 2025Enhancing Domain-Specific Encoder Models with LLM-Generated Data: How to Leverage Ontologies, and How to Do Without Themcited by 4
  25. 2026Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stagescited by 4
  26. 2026Reasoning-Driven Synthetic Data Generation and Evaluationcited by 4
  27. 2024SoftSRV: Learn to Generate Targeted Synthetic Datacited by 3
  28. 2025Scaling LLM Pre-training with Vocabulary Curriculumcited by 3
  29. 2025LM-mixup: Text Data Augmentation via Language Model based Mixupcited by 3
  30. 2024Iterative Graph Alignmentcited by 2
  31. 2024Error-driven Data-efficient Large Multimodal Model Tuningcited by 2
  32. 2025Learning from Reasoning Failures via Synthetic Data Generationcited by 2
  33. 2025Large Language models for Time Series Analysis: Techniques, Applications, and Challengescited by 2
  34. 2026Socratic-Geo: Synthetic Data Generation and Geometric Reasoning via Multi-Agent Interactioncited by 2
  35. 2024Cognitively Inspired Energy-Based World Modelscited by 1
  36. 2024Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompressioncited by 1
  37. 2024Unsupervised Data Validation Methods for Efficient Model Trainingcited by 1
  38. 2025A Vision-Language Pre-training Model-Guided Approach for Mitigating Backdoor Attacks in Federated Learningcited by 1

50 most cited of 69 All 69 (JSON)