mlsanityData to 7 Oct 2026

Data Diversity Matters for Robust Instruction Tuning

arXiv 2311.14736 ↗First posted 21 Nov 2023cs.CL cs.LGcited by 76

Not on the map. Why?

Abstract (arXiv)

Recent works have shown that by curating high quality and diverse instruction tuning datasets, we can significantly improve instruction-following capabilities. However, creating such datasets is difficult and most works rely on manual curation or proprietary language models. Automatic data curation is difficult as it is still not clear how we can define diversity for instruction tuning, how diversity and quality depend on one other, and how we can optimize dataset quality and diversity. To resolve these issue, we propose a new algorithm, Quality-Diversity Instruction Tuning (QDIT). QDIT provides a simple method to simultaneously control dataset diversity and quality, allowing us to conduct an in-depth study on the effect of diversity and quality on instruction tuning performance. From this study we draw two key insights (1) there is a natural tradeoff between data diversity and quality and (2) increasing data diversity significantly improves the worst case instruction following performance, therefore improving robustness. We validate the performance of QDIT on several large scale instruction tuning datasets, where we find it can substantially improve worst and average case performance compared to quality-driven data selection.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 43 of its 63 references that are in the map. First posted on arXiv 21 Nov 2023.

Builds on

43 of 63 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  11. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  12. 2023Mistral 7Bcited by 3,065
31 more
  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  6. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  7. 2023LIMA: Less Is More for Alignmentcited by 1,080
  8. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  9. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  10. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  11. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  12. 2017Crowdsourcing Multiple Choice Science Questionscited by 742
  13. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  14. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 696
  15. 2023Instruction Tuning with GPT-4cited by 676
  16. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  17. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  18. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  19. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 301
  20. 2023The False Promise of Imitating Proprietary LLMscited by 254
  21. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 227
  22. 2018Constituency Parsing with a Self-Attentive Encodercited by 196
  23. 2023Self-Alignment with Instruction Backtranslationcited by 149
  24. 2023D4: Improving LLM Pretraining via Document De-Duplication and Diversificationcited by 146
  25. 2023HelpSteer: Multi-attribute Helpfulness Dataset for SteerLMcited by 140
  26. 2018Multilingual Constituency Parsing with Self-Attention and Pre-Trainingcited by 117
  27. 2023SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHFcited by 107
  28. 2014Lazier Than Lazy Greedycited by 97
  29. 2023D2 Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruningcited by 68
  30. 2023INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Modelscited by 62
  31. 2023Accelerating Batch Active Learning Using Continual Learning Techniquescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 21; 2025: 33; 2026: 21.

76 papers.

Sort:

By area:

  1. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  2. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 172
  3. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 134
  4. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 31
  5. 2024A Survey of Table Reasoning with Large Language Modelscited by 29
  6. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  7. 2024Diversity Measurement and Subset Selection for Instruction Tuning Datasetscited by 28
  8. 2024TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Datacited by 28
  9. 2024ICONS: Influence Consensus for Vision-Language Data Selectioncited by 28
  10. 2024HARP: A challenging human-annotated math reasoning benchmarkcited by 25
  11. 2025Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metriccited by 24
  12. 2024An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Modelscited by 22
38 more
  1. 2025Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilitiescited by 22
  2. 2025Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learningcited by 19
  3. 2023Data Management For Training Large Language Models: A Surveycited by 17
  4. 2024Instruction Diversity Drives Generalization To Unseen Taskscited by 15
  5. 2024CodeACT: Code Adaptive Compute-efficient Tuning Framework for Code LLMscited by 15
  6. 2024DELIFT: Data Efficient Language model Instruction Fine Tuningcited by 14
  7. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 14
  8. 2024Diversify and Conquer: Diversity-Centric Data Selection with Iterative Refinementcited by 13
  9. 2025LEAD: Iterative Data Selection for Efficient LLM Instruction Tuningcited by 13
  10. 2024Mini-batch Coresets for Memory-efficient Language Model Training on Data Mixturescited by 12
  11. 2025MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Spacecited by 12
  12. 2025Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencodercited by 11
  13. 2024Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuningcited by 9
  14. 2025How Diversely Can Language Models Solve Problems? Exploring the Algorithmic Diversity of Model-Generated Codecited by 9
  15. 2024Semi-supervised Fine-tuning for Large Language Modelscited by 8
  16. 2025NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmarkcited by 7
  17. 2025Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoningcited by 7
  18. 2025Data Selection for Fine-tuning Vision Language Models via Cross Modal Alignment Trajectoriescited by 7
  19. 2025Importance-Aware Data Selection for Efficient LLM Instruction Tuningcited by 7
  20. 2024SCAR: Data Selection via Style Consistency-Aware Response Ranking for Efficient Instruction-Tuning of Large Language Modelscited by 6
  21. 2024Multilingual Arbitrage: Optimizing Data Pools to Accelerate Multilingual Progresscited by 6
  22. 2025CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdomcited by 6
  23. 2025TreeSynth: Synthesizing Diverse Data from Scratch via Tree-Guided Subspace Partitioningcited by 6
  24. 2025Data-efficient LLM Fine-tuning for Code Generationcited by 5
  25. 2025EDGE: Efficient Data Selection for LLM Agents via Guideline Effectivenesscited by 4
  26. 2025ArgInstruct: Specialized Instruction Fine-Tuning for Computational Argumentationcited by 4
  27. 2025KORMo: Korean Open Reasoning Model for Everyonecited by 4
  28. 2026Enhancing Instruction Following of LLMs via Activation Steering with Dynamic Rejectioncited by 4
  29. 2025From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuningcited by 3
  30. 2025ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuningcited by 3
  31. 2026Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methodscited by 3
  32. 2025RICo: Refined In-Context Contribution for Automatic Instruction-Tuning Data Selectioncited by 2
  33. 2025We Need to Measure Data Diversity in NLP -- Better and Broadercited by 2
  34. 2025Exploring Solution Divergence and Its Effect on Large Language Model Problem Solvingcited by 2
  35. 2025GradientSpace: Unsupervised Data Clustering for Improved Instruction Tuningcited by 2
  36. 2024Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognitioncited by 1
  37. 2025LASER: Stratified Selective Sampling for Instruction Tuning with Dedicated Scoring Strategycited by 1
  38. 2025Improving Task Diversity in Label Efficient Supervised Finetuning of LLMscited by 1

50 most cited of 76 All 76 (JSON)