mlsanityData to 7 Oct 2026

LESS: Selecting Influential Data for Targeted Instruction Tuning

arXiv 2402.04333 ↗First posted 6 Feb 2024cs.CL cs.AI cs.LGcited by 514

Not on the map. Why?

Cited by 60 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Instruction tuning has unlocked powerful capabilities in large language models (LLMs), effectively using combined datasets to develop generalpurpose chatbots. However, real-world applications often require a specialized suite of skills (e.g., reasoning). The challenge lies in identifying the most relevant data from these extensive datasets to effectively develop specific capabilities, a setting we frame as targeted instruction tuning. We propose LESS, an optimizer-aware and practically efficient algorithm to effectively estimate data influences and perform Low-rank gradiEnt Similarity Search for instruction data selection. Crucially, LESS adapts existing influence formulations to work with the Adam optimizer and variable-length instruction data. LESS first constructs a highly reusable and transferable gradient datastore with low-dimensional gradient features and then selects examples based on their similarity to few-shot examples embodying a specific capability. Experiments show that training on a LESS-selected 5% of the data can often outperform training on the full dataset across diverse downstream tasks. Furthermore, the selected data is highly transferable: smaller models can be leveraged to select useful data for larger models and models from different families. Our qualitative analysis shows that our method goes beyond surface form cues to identify data that exemplifies the necessary reasoning skills for the intended downstream application.

Key facts. Cited by 514 papers in the map (data through 7 Oct 2026). Builds on 65 of its 88 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

65 of 88 references.

References · 65

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2023Mistral 7Bcited by 3,065
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  11. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  12. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
38 more
  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  2. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  3. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  4. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  5. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  6. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  7. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  8. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  9. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
  10. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  11. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  12. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  13. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  14. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
  15. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  16. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
  17. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  18. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  19. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  20. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 405
  21. 2023Orca: Progressive Learning from Complex Explanation Traces of GPT-4cited by 369
  22. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 354
  23. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 337
  24. 2023Data Selection for Language Models via Importance Resamplingcited by 322
  25. 2021GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Trainingcited by 269
  26. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 251
  27. 2020Influence Functions in Deep Learning Are Fragilecited by 247
  28. 2022Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learntcited by 233
  29. 2020Dataset Meta-Learning from Kernel Ridge-Regressioncited by 230
  30. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  31. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  32. 2023Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learningcited by 149
  33. 2022If Influence Functions are the Answer, Then What is the Question?cited by 142
  34. 2023Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Modelscited by 120
  35. 2022A Kernel-Based View of Language Model Fine-Tuningcited by 118
  36. 2023MoDS: Model-oriented Data Selection for Instruction Tuningcited by 107
  37. 2022On the SDEs and Scaling Rules for Adaptive Gradient Algorithmscited by 100
  38. 2022More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalizecited by 87

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 115; 2025: 203; 2026: 196.

514 papers.

Sort:

By area:

  1. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  2. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  3. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 338
  4. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 288
  5. 2024NVILA: Efficient Frontier Visual Language Modelscited by 206
  6. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 172
  7. 2024Rho-1: Not All Tokens Are What You Needcited by 132
  8. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
  9. 2025Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rolloutscited by 108
  10. 2024MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Modelscited by 96
  11. 2024Learning Dynamics of LLM Finetuningcited by 93
  12. 2024DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solvingcited by 92
38 more
  1. 2024What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functionscited by 88
  2. 2024Data Shapley in One Training Runcited by 76
  3. 2024A Survey of Multimodal Large Language Model from A Data-centric Perspectivecited by 73
  4. 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
  5. 2025Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-trainingcited by 62
  6. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 60
  7. 2025PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selectioncited by 57
  8. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 55
  9. 2024Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimationcited by 53
  10. 2025The Best Instruction-Tuning Data are Those That Fitcited by 48
  11. 2024Adaptive Data Optimization: Dynamic Sample Selection with Scaling Lawscited by 46
  12. 2024JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Modelscited by 44
  13. 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMscited by 44
  14. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  15. 2025GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolutioncited by 41
  16. 2024Scalable Influence and Fact Tracing for Large Language Model Pretrainingcited by 40
  17. 2025Prompt Curriculum Learning for Efficient LLM Post-Trainingcited by 40
  18. 2025Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuningcited by 39
  19. 2024Data Selection via Optimal Control for Language Modelscited by 38
  20. 2025Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoningcited by 38
  21. 2024ScaleBiO: Scalable Bilevel Optimization for LLM Data Reweightingcited by 35
  22. 2024SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflectioncited by 34
  23. 2024Harnessing Diversity for Important Data Selection in Pretraining Large Language Modelscited by 34
  24. 2025Large-Scale Data Selection for Instruction Tuningcited by 34
  25. 2024Training Data Attribution via Approximate Unrolled Differentiationcited by 33
  26. 2024Aioli: A Unified Optimization Framework for Language Model Data Mixingcited by 33
  27. 2024G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translationcited by 29
  28. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  29. 2024TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Datacited by 28
  30. 2024ICONS: Influence Consensus for Vision-Language Data Selectioncited by 28
  31. 2024I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigmcited by 27
  32. 2024Improving Data Efficiency via Curating LLM-Driven Rating Systemscited by 26
  33. 2024Hybrid Preferences: Learning to Route Instances for Human vs. AI Feedbackcited by 25
  34. 2025SimPER: A Minimalist Approach to Preference Alignment without Hyperparameterscited by 25
  35. 2025DataMIL: Selecting Data for Robot Imitation Learning with Datamodelscited by 25
  36. 2024Concept-skill Transferability-based Data Selection for Large Vision-Language Modelscited by 24
  37. 2024ProSec: Fortifying Code LLMs with Proactive Security Alignmentcited by 24
  38. 2024How to Synthesize Text Data without Model Collapse?cited by 24

50 most cited of 514 All 514 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Aligning Instruction Tuning with Pre-training
  2. 2023Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
  3. 2026Synthesizing Instruction-Tuning Datasets with Contrastive Decoding
  4. 2024MAmmoTH2: Scaling Instructions from the Web
  5. 2023Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks
  6. 2023Tuna: Instruction Tuning using Feedback from Large Language Models
  7. 2024CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions
  8. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models