mlsanityData to 7 Oct 2026

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition

arXiv 2310.05492 ↗First posted 9 Oct 2023cs.CL cs.AI cs.LGcited by 222

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large language models (LLMs) with enormous pre-training tokens and parameters emerge diverse abilities, including math reasoning, code generation, and instruction following. These abilities are further enhanced by supervised fine-tuning (SFT). While the open-source community has explored ad-hoc SFT for enhancing individual capabilities, proprietary LLMs exhibit versatility across various skills. Therefore, understanding the facilitation of multiple abilities via SFT is paramount. In this study, we specifically focuses on the interplay of data composition between mathematical reasoning, code generation, and general human-aligning abilities during SFT. We propose four intriguing research questions to explore the association between model performance and various factors including data amount, composition ratio, model size and SFT strategies. Our experiments reveal that distinct capabilities scale differently and larger models generally show superior performance with same amount of data. Mathematical reasoning and code generation consistently improve with increasing data amount, whereas general abilities plateau after roughly a thousand samples. Moreover, we observe data composition appears to enhance various abilities under limited data conditions, yet can lead to performance conflicts when data is plentiful. Our findings also suggest the amount of composition data influences performance more than the composition ratio. In analysis of SFT strategies, we find that sequentially learning multiple skills risks catastrophic forgetting. Our proposed Dual-stage Mixed Fine-tuning (DMT) strategy offers a promising solution to learn multiple abilities with different scaling patterns.

Key facts. Cited by 222 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 37 of its 84 references that are in the map. First posted on arXiv 9 Oct 2023.

Builds on

37 of 84 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023PaLM 2 Technical Report17 May
  2. 2023LLaMA: Open and Efficient Foundation Language Models27 Feb
  3. 2022Towards Reasoning in Large Language Models: A Survey20 Dec
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Models6 Oct

Other references · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
21 more
  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  2. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  3. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  4. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  5. 2023LIMA: Less Is More for Alignmentcited by 1,080
  6. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  7. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  8. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  9. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 483
  10. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  11. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  12. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  13. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  14. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  15. 2022Unified Structure Generation for Universal Information Extractioncited by 220
  16. 2021R-Drop: Regularized Dropout for Neural Networkscited by 202
  17. 2021Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuningcited by 145
  18. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  19. 2019An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Modelscited by 39
  20. 2021Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learningcited by 35
  21. 2022HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation Perturbationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 79; 2025: 82; 2026: 55.

222 papers.

Sort:

By area:

  1. 2024Qwen2 Technical Reportcited by 2,076
  2. 2024Yi: Open Foundation Models by 01.AIcited by 706
  3. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  4. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 423
  5. 2025Agentic Reinforced Policy Optimizationcited by 166
  6. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  7. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  8. 2024Continual Learning for Large Language Models: A Surveycited by 144
  9. 2024Common 7B Language Models Already Possess Strong Math Capabilitiescited by 108
  10. 2024LLaMA Pro: Progressive LLaMA with Block Expansioncited by 99
  11. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  12. 2024Small LLMs Are Weak Tool Learners: A Multi-LLM Agentcited by 92
38 more
  1. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 80
  2. 2024Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Modelscited by 73
  3. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  4. 2024A Survey on Employing Large Language Models for Text-to-SQL Taskscited by 67
  5. 2023DrugAssist: A Large Language Model for Molecule Optimizationcited by 57
  6. 2024RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesiscited by 55
  7. 2024Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimationcited by 53
  8. 2024Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Trainingcited by 53
  9. 2025Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Modelscited by 51
  10. 2025Multi-Attribute Steering of Language Models via Targeted Interventioncited by 44
  11. 2024WARP: On the Benefits of Weight Averaged Rewarded Policiescited by 42
  12. 2025Parametric Retrieval Augmented Generationcited by 38
  13. 2024SciRIFF: A Resource to Enhance Language Model Instruction-Following over Scientific Literaturecited by 37
  14. 2024Weak-to-Strong Reasoningcited by 32
  15. 2024Evaluating Language Models as Synthetic Data Generatorscited by 32
  16. 2024Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMscited by 31
  17. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 31
  18. 2024CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Masterycited by 30
  19. 2024CityGPT: Empowering Urban Spatial Cognition of Large Language Modelscited by 29
  20. 2024Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignmentcited by 28
  21. 2024Discerning and Resolving Knowledge Conflicts through Adaptive Decoding with Contextual Information-Entropy Constraintcited by 27
  22. 2024Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generationcited by 27
  23. 2023Evidence of interrelated cognitive-like capabilities in large language models: Indications of artificial general intelligence or achievement?cited by 26
  24. 2023On Diversified Preferences of Large Language Model Alignmentcited by 26
  25. 2024LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Trainingcited by 25
  26. 2025GRAM: A Generative Foundation Reward Model for Reward Generalizationcited by 25
  27. 2023LARP: Language-Agent Role Play for Open-World Gamescited by 24
  28. 2023Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Taskcited by 23
  29. 2025Fine-Grained GRPO for Precise Preference Alignment in Flow Modelscited by 21
  30. 2024Extending LLMs' Context Window with 100 Samplescited by 20
  31. 2024Aloe: A Family of Fine-tuned Open Healthcare LLMscited by 20
  32. 2024RetroLLM: Empowering Large Language Models to Retrieve Fine-grained Evidence within Generationcited by 20
  33. 2025MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuningcited by 19
  34. 2024HFT: Half Fine-Tuning for Large Language Modelscited by 18
  35. 2025LoRI: Reducing Cross-Task Interference in Multi-Task Low-Rank Adaptationcited by 18
  36. 2023Data Management For Training Large Language Models: A Surveycited by 17
  37. 2024SMART: Submodular Data Mixture Strategy for Instruction Tuningcited by 16
  38. 2024Prompt-based Code Completion via Multi-Retrieval Augmented Generationcited by 16

50 most cited of 222 All 222 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Unveiling the Impact of Coding Data Instruction Fine-Tuning on Large Language Models Reasoning
  2. 2023Dynamics of Instruction Fine-Tuning for Chinese Large Language Models
  3. 2024Revisiting the Superficial Alignment Hypothesis
  4. 2026Not All Code Is Equal: A Data-Centric Study of Code Complexity and LLM Reasoning
  5. 2025Climbing the Ladder of Reasoning: What LLMs Can-and Still Can't-Solve after SFT?
  6. 2024Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability
  7. 2026Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarks
  8. 2024An Empirical Study of Data Ability Boundary in LLMs' Math Reasoning