mlsanityData to 7 Oct 2026

Diversity Measurement and Subset Selection for Instruction Tuning Datasets

arXiv 2402.02318 ↗First posted 4 Feb 2024cs.LG cs.CLcited by 28

Not on the map. Why?

Abstract (arXiv)

We aim to select data subsets for the fine-tuning of large language models to more effectively follow instructions. Prior work has emphasized the importance of diversity in dataset curation but relied on heuristics such as the number of tasks. In this paper, we use determinantal point processes to capture the diversity and quality of instruction tuning datasets for subset selection. We propose to measure dataset diversity with log determinant distance that is the distance between the dataset of interest and a maximally diverse reference dataset. Our experiments demonstrate that the proposed diversity measure in the normalized weight gradient space is correlated with downstream instruction-following performance. Consequently, it can be used to inform when data selection is the most helpful and to analyze dataset curation strategies. We demonstrate the utility of our approach on various instruction tuning datasets.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 45 of its 73 references that are in the map. First posted on arXiv 4 Feb 2024.

Builds on

45 of 73 references.

References · 45

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  12. 2020Learning to summarize from human feedbackcited by 2,562
33 more
  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  2. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  3. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  4. 2023LIMA: Less Is More for Alignmentcited by 1,080
  5. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  7. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  8. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  9. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  10. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  11. 2014Distributed Representations of Sentences and Documentscited by 677
  12. 2023Instruction Tuning with GPT-4cited by 676
  13. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  14. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  15. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  16. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
  17. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  18. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  19. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  20. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  21. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  22. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  23. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 354
  24. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 301
  25. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  26. 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 295
  27. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  28. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 227
  29. 2023#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Modelscited by 108
  30. 2023Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedbackcited by 78
  31. 2023Data Diversity Matters for Robust Instruction Tuningcited by 76
  32. 2016Active Learning for Speech Recognition: the Power of Gradientscited by 25
  33. 2024An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Modelscited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 13; 2026: 12.

28 papers.

Sort:

By area:

  1. 2025The Best Instruction-Tuning Data are Those That Fitcited by 48
  2. 2025Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metriccited by 24
  3. 2025VoiceAgentBench: Are Voice Assistants ready for agentic tasks?cited by 15
  4. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 14
  5. 2024Diversify and Conquer: Diversity-Centric Data Selection with Iterative Refinementcited by 13
  6. 2025MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Spacecited by 12
  7. 2024API-guided Dataset Synthesis to Finetune Large Code Modelscited by 11
  8. 2025DUET: Optimizing Training Data Mixtures via Feedback from Unseen Evaluation Taskscited by 6
  9. 2025CrowdSelect: Synthetic Instruction Data Selection with Multi-LLM Wisdomcited by 6
  10. 2025The Role of Diversity in In-Context Learning for Large Language Modelscited by 6
  11. 2025Skill-Targeted Adaptive Trainingcited by 6
  12. 2026Detecting and Suppressing Reward Hacking with Gradient Fingerprintscited by 4
16 more
  1. 2025From Macro to Micro: Probing Dataset Diversity in Language Model Fine-Tuningcited by 3
  2. 2024COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptationcited by 2
  3. 2026Cram Less to Fit More: Training Data Pruning Improves Memorization of Factscited by 2
  4. 2025Improving Task Diversity in Label Efficient Supervised Finetuning of LLMscited by 1
  5. 2025Are LLMs Enough for Hyperpartisan, Fake, Polarized and Harmful Content Detection? Evaluating In-Context Learning vs. Fine-Tuningcited by 1
  6. 2026UpSkill: Mutual Information Skill Learning for Structured Response Diversity in LLMscited by 1
  7. 2026Does the Question Really Matter? Training-Free Data Selection for Vision-Language SFTcited by 1
  8. 2026DataProphet: Demystifying Supervision Data Generalization in Multimodal LLMscited by 1
  9. 2026KrishokChat: A Provenance-Traceable Multi-Task Bengali Agricultural Benchmark with Safety-Critical Chemical Advisorycited by 1
  10. 2026emb-diversity: A Tool for Embedding-Based Measurement of Data Diversitycited by 1
  11. 2025VOYAGER: A Training Free Approach for Generating Diverse Datasets using LLMscited by 0
  12. 2026The Chicken and Egg Dilemma: Co-optimizing Data and Model Configurations for LLMscited by 0
  13. 2026Optimizing What We Trust: Reliability-Guided QUBO Selection of Multi-Agent Weak Framing Signals for Arabic Sentiment Predictioncited by 0
  14. 2026EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptationcited by 0
  15. 2026InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memoriescited by 0
  16. 2026Towards Scalable Data Diversification for Language Model Pretraining via Leverage Score Samplingcited by 0