mlsanityData to 7 Oct 2026

Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models

arXiv 2408.02085 ↗First posted 4 Aug 2024cs.CV cs.AI cs.CLcited by 29

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Instruction tuning plays a critical role in aligning large language models (LLMs) with human preference. Despite the vast amount of open instruction datasets, naively training a LLM on all existing instructions may not be optimal and practical. To pinpoint the most beneficial datapoints, data assessment and selection methods have been proposed in the fields of natural language processing (NLP) and deep learning. However, under the context of instruction tuning, there still exists a gap in knowledge on what kind of data evaluation metrics can be employed and how they can be integrated into the selection mechanism. To bridge this gap, we present a comprehensive review on existing literature of data assessment and selection especially for instruction tuning of LLMs. We systematically categorize all applicable methods into quality-based, diversity-based, and importance-based ones where a unified, fine-grained taxonomy is structured. For each category, representative methods are elaborated to describe the landscape of relevant research. In addition, comparison between the latest methods is conducted on their officially reported results to provide in-depth discussions on their limitations. Finally, we summarize the open challenges and propose the promosing avenues for future studies. All related contents are available at https://github.com/yuleiqin/fantastic-data-engineering.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 210 of its 387 references that are in the map. First posted on arXiv 4 Aug 2024.

Builds on

210 of 387 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing12 Jun
  2. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning25 Dec
  3. 2023LIMA: Less Is More for Alignment18 May
  4. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignment14 Apr
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec
  6. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 204

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  9. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  10. 2020Scaling Laws for Neural Language Modelscited by 6,411
  11. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
38 more
  1. 2023Qwen Technical Reportcited by 3,273
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2024Qwen2 Technical Reportcited by 2,076
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  6. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  7. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  8. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  9. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  10. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  11. 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
  12. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  13. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  14. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  15. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  16. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  17. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  18. 2023StarCoder: may the source be with you!cited by 790
  19. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  20. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  21. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  22. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  23. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  24. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  25. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  26. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  27. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  28. 2020Language-agnostic BERT Sentence Embeddingcited by 621
  29. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  30. 2023Instruction Tuning for Large Language Models: A Surveycited by 617
  31. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  32. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
  33. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  34. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 514
  35. 2020Dataset Condensation with Gradient Matchingcited by 510
  36. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  37. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  38. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452

50 most cited of 204 All 204 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 15; 2026: 7.

29 papers.

Sort:

By area:

  1. 2024HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systemscited by 37
  2. 2025Principled Data Selection for Alignment: The Hidden Risks of Difficult Examplescited by 37
  3. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 31
  4. 2025Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metriccited by 24
  5. 2024Rethinking Data Selection at Scale: Random Selection is Almost All You Needcited by 22
  6. 2025Improving Influence-based Instruction Tuning Data Selection for Balanced Learning of Diverse Capabilitiescited by 22
  7. 2025The Rise of Small Language Models in Healthcare: A Comprehensive Surveycited by 18
  8. 2025MCIF: Multimodal Crosslingual Instruction-Following Benchmark from Scientific Talkscited by 14
  9. 2024Chasing Random: Instruction Selection Strategies Fail to Generalizecited by 12
  10. 2025Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learningcited by 10
  11. 2025Importance-Aware Data Selection for Efficient LLM Instruction Tuningcited by 7
  12. 2024Instruction Tuning Vs. In-Context Learning: Revisiting Large Language Models in Few-Shot Computational Social Sciencecited by 4
17 more
  1. 2025Towards Understanding Valuable Preference Data for Large Language Model Alignmentcited by 4
  2. 2024Fast Training Dataset Attribution via In-Context Learningcited by 2
  3. 2024Leveraging Open Knowledge for Advancing Task Expertise in Large Language Modelscited by 2
  4. 2025An Index-based Approach for Efficient and Effective Web Content Extractioncited by 2
  5. 2026DataChef: Cooking Up Optimal Data Recipes for LLM Adaptation via Reinforcement Learningcited by 2
  6. 2025Cream of the Crop: Harvesting Rich, Scalable and Transferable Multi-Modal Data for Instruction Fine-Tuningcited by 1
  7. 2025Improving Task Diversity in Label Efficient Supervised Finetuning of LLMscited by 1
  8. 2026Neuron-Aware Data Selection In Instruction Tuning For Large Language Modelscited by 1
  9. 2026Synthetic Data Generation for Training Diversified Commonsense Reasoning Modelscited by 1
  10. 2026Filter-then-Weight: Online Data Selection and Reweighting for LLM Fine-Tuningcited by 1
  11. 2026SFGA: A Statistics-First Gating Architecture with Adjudicative Escalation for Trustworthy SFT Data Procurementcited by 1
  12. 2025InsBank: Evolving Instruction Subset for Ongoing Alignmentcited by 0
  13. 2025Towards Automatic Continual Learning: A Self-Adaptive Framework for Continual Instruction Tuningcited by 0
  14. 2025Team QUST at SemEval-2025 Task 10: Evaluating Large Language Models in Multiclass Multi-label Classification of News Entity Framingcited by 0
  15. 2025Accelerate Scaling of LLM Finetuning via Quantifying the Coverage and Depth of Instruction Setcited by 0
  16. 2026MADS: Model-Aware Diverse Core Set Selection for Instruction Tuningcited by 0
  17. 2026Aligning Human Sense: Calibrated Distributional Reward Learning for Video Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
  2. 2025Aligning Instruction Tuning with Pre-training
  3. 2023Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuning
  4. 2025LEAD: Iterative Data Selection for Efficient LLM Instruction Tuning
  5. 2024Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models
  6. 2024ROSE: A Reward-Oriented Data Selection Framework for LLM Task-Specific Instruction Tuning
  7. 2024Enhancing and Assessing Instruction-Following with Fine-Grained Instruction Variants
  8. 2023Dynamics of Instruction Fine-Tuning for Chinese Large Language Models