mlsanityData to 7 Oct 2026

I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses

arXiv 2402.11192 ↗First posted 17 Feb 2024cs.CL cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

This paper explores an intriguing observation: fine-tuning a large language model (LLM) with responses generated by a LLM often yields better results than using responses generated by humans, particularly in reasoning tasks. We conduct an in-depth investigation to understand why this occurs. Contrary to the common belief that these instances is due to the more detailed nature of LLM-generated content, our study identifies another contributing factor: an LLM is inherently more "familiar" with LLM generated responses. This familiarity is evidenced by lower perplexity before fine-tuning. We design a series of experiments to understand the impact of the "familiarity" and our conclusion reveals that this "familiarity" significantly impacts learning performance. Training with LLM-generated responses not only enhances performance but also helps maintain the model's capabilities in other reasoning tasks after fine-tuning on a specific task.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 30 of its 74 references that are in the map. First posted on arXiv 17 Feb 2024.

Builds on

30 of 74 references.

References · 30

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2021Program Synthesis with Large Language Modelscited by 3,509
  8. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  9. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  10. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  11. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  12. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
18 more
  1. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  2. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  3. 2023Detecting Language Model Attacks with Perplexitycited by 411
  4. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  5. 2022Large Language Models Are Reasoning Teacherscited by 318
  6. 2022Teaching Small Language Models to Reasoncited by 279
  7. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  8. 2022Demystifying Prompts in Language Models via Perplexity Estimationcited by 215
  9. 2020A Systematic Assessment of Syntactic Generalization in Neural Language Modelscited by 163
  10. 2023AugGPT: Leveraging ChatGPT for Text Data Augmentationcited by 128
  11. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 123
  12. 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Taskscited by 62
  13. 2023The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Taskscited by 35
  14. 2023ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPTcited by 32
  15. 2024Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Modelscited by 32
  16. 2022BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Samplingcited by 30
  17. 2022QAmeleon: Multilingual QA with Only 5 Examplescited by 25
  18. 2021Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classificationcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 9; 2026: 3.

13 papers.

Sort:

By area:

  1. 2025Self-Training Elicits Concise Reasoning in Large Language Modelscited by 97
  2. 2025DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignmentcited by 47
  3. 2025LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perceptioncited by 14
  4. 2025Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learningcited by 12
  5. 2025Position Bias Mitigates Position Bias:Mitigate Position Bias Through Inter-Position Knowledge Distillationcited by 8
  6. 2024Boosting LLM Translation Skills without General Ability Loss via Rationale Distillationcited by 5
  7. 2025Enhancing Domain-Specific Encoder Models with LLM-Generated Data: How to Leverage Ontologies, and How to Do Without Themcited by 4
  8. 2025Menta: A Small Language Model for On-Device Mental Health Predictioncited by 3
  9. 2025Efficient Response Generation Strategy Selection for Fine-Tuning Large Language Models Through Self-Aligned Perplexitycited by 1
  10. 2025What Matters in LLM-generated Data: Diversity and Its Effect on Model Fine-Tuningcited by 1
  11. 2026Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillationcited by 1
  12. 2026Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuningcited by 0
1 more
  1. 2026Decoupling Token Roles in Autoregressive Pretrainingcited by 0