# I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses

arXiv: https://arxiv.org/abs/2402.11192 · HTML: https://mlsanity.com/paper/2402.11192/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 74 references that are in the map.
- First posted on arXiv 17 Feb 2024.

## Builds on (most cited first, 30 of 30)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [Reinforced Self-Training (ReST) for Language Modeling](https://mlsanity.com/paper/2308.08998/) (2023)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models](https://mlsanity.com/paper/2312.06585/) (2023)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [Teaching Small Language Models to Reason](https://mlsanity.com/paper/2212.08410/) (2022)
- [Specializing Smaller Language Models towards Multi-Step Reasoning](https://mlsanity.com/paper/2301.12726/) (2023)
- [Demystifying Prompts in Language Models via Perplexity Estimation](https://mlsanity.com/paper/2212.04037/) (2022)
- [A Systematic Assessment of Syntactic Generalization in Neural Language Models](https://mlsanity.com/paper/2005.03692/) (2020)
- [AugGPT: Leveraging ChatGPT for Text Data Augmentation](https://mlsanity.com/paper/2302.13007/) (2023)
- [Explanations from Large Language Models Make Small Reasoners Better](https://mlsanity.com/paper/2210.06726/) (2022)
- [Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks](https://mlsanity.com/paper/2305.18395/) (2023)
- [The Parrot Dilemma: Human-Labeled vs. LLM-augmented Data in Classification Tasks](https://mlsanity.com/paper/2304.13861/) (2023)
- [ZeroShotDataAug: Generating and Augmenting Training Data with ChatGPT](https://mlsanity.com/paper/2304.14334/) (2023)
- [Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models](https://mlsanity.com/paper/2402.10430/) (2024)
- [BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Sampling](https://mlsanity.com/paper/2207.06814/) (2022)
- [QAmeleon: Multilingual QA with Only 5 Examples](https://mlsanity.com/paper/2211.08264/) (2022)
- [Guiding Generative Language Models for Data Augmentation in Few-Shot Text Classification](https://mlsanity.com/paper/2111.09064/) (2021)

## Built on by (most cited first, 15 of 15)

- [Self-Training Elicits Concise Reasoning in Large Language Models](https://mlsanity.com/paper/2502.20122/) (2025)
- [DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment](https://mlsanity.com/paper/2507.02768/) (2025)
- [LongPerceptualThoughts: Distilling System-2 Reasoning for System-1 Perception](https://mlsanity.com/paper/2504.15362/) (2025)
- [Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learning](https://mlsanity.com/paper/2501.14315/) (2025)
- [Position Bias Mitigates Position Bias:Mitigate Position Bias Through Inter-Position Knowledge Distillation](https://mlsanity.com/paper/2508.15709/) (2025)
- [Boosting LLM Translation Skills without General Ability Loss via Rationale Distillation](https://mlsanity.com/paper/2410.13944/) (2024)
- [Enhancing Domain-Specific Encoder Models with LLM-Generated Data: How to Leverage Ontologies, and How to Do Without Them](https://mlsanity.com/paper/2503.22006/) (2025)
- [Menta: A Small Language Model for On-Device Mental Health Prediction](https://mlsanity.com/paper/2512.02716/) (2025)
- [Efficient Response Generation Strategy Selection for Fine-Tuning Large Language Models Through Self-Aligned Perplexity](https://mlsanity.com/paper/2502.11779/) (2025)
- [What Matters in LLM-generated Data: Diversity and Its Effect on Model Fine-Tuning](https://mlsanity.com/paper/2506.19262/) (2025)
- [Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation](https://mlsanity.com/paper/2606.00628/) (2026)
- [Data Difficulty and the Generalization--Extrapolation Tradeoff in LLM Fine-Tuning](https://mlsanity.com/paper/2605.12906/) (2026)
- [Decoupling Token Roles in Autoregressive Pretraining](https://mlsanity.com/paper/2609.33405/) (2026)
- [Selecting Long-Horizon Trajectories for Reliable and Efficient Terminal-Agent Training](https://mlsanity.com/paper/2610.05831/) (2026)
- [Teaching Agents in their Own Style: Blackbox Teacher Distillation for Web Agents](https://mlsanity.com/paper/2610.19834/) (2026)

## Abstract (arXiv)

This paper explores an intriguing observation: fine-tuning a large language model (LLM) with responses generated by a LLM often yields better results than using responses generated by humans, particularly in reasoning tasks. We conduct an in-depth investigation to understand why this occurs. Contrary to the common belief that these instances is due to the more detailed nature of LLM-generated content, our study identifies another contributing factor: an LLM is inherently more "familiar" with LLM generated responses. This familiarity is evidenced by lower perplexity before fine-tuning. We design a series of experiments to understand the impact of the "familiarity" and our conclusion reveals that this "familiarity" significantly impacts learning performance. Training with LLM-generated responses not only enhances performance but also helps maintain the model's capabilities in other reasoning tasks after fine-tuning on a specific task.