mlsanityData to 7 Oct 2026

Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning

arXiv 2402.00530 ↗First posted 1 Feb 2024cs.CLcited by 134

Not on the map. Why?

Abstract (arXiv)

Instruction tuning is critical to improve LLMs but usually suffers from low-quality and redundant data. Data filtering for instruction tuning has proved important in improving both the efficiency and performance of the tuning process. But it also leads to extra cost and computation due to the involvement of LLMs in this process. To reduce the filtering cost, we study Superfiltering: Can we use a smaller and weaker model to select data for finetuning a larger and stronger model? Despite the performance gap between weak and strong language models, we find their highly consistent capability to perceive instruction difficulty and data selection results. This enables us to use a much smaller and more efficient model to filter the instruction data used to train a larger language model. Not only does it largely speed up the data filtering, but the filtered-data-finetuned LLM achieves even better performance on standard benchmarks. Extensive experiments validate the efficacy and efficiency of our approach.

Key facts. Cited by 134 papers in the map (data through 7 Oct 2026). Builds on 38 of its 55 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

38 of 55 references.

References · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  11. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  12. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
26 more
  1. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  2. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  3. 2023LIMA: Less Is More for Alignmentcited by 1,080
  4. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  5. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  6. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
  7. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  8. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  9. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  10. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  11. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  12. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 395
  13. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 337
  14. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 301
  15. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
  16. 2023MMC: Advancing Multimodal Chart Understanding with Large-scale Instruction Tuningcited by 149
  17. 2021CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLPcited by 142
  18. 2023MoDS: Model-oriented Data Selection for Instruction Tuningcited by 107
  19. 2020Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?cited by 95
  20. 2020Look at the First Sentence: Position Bias in Question Answeringcited by 83
  21. 2023Data Diversity Matters for Robust Instruction Tuningcited by 76
  22. 2023MM-LIMA: Less Is More for Alignment in Multi-Modal Datasetscited by 61
  23. 2023Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Taskscited by 53
  24. 2020Scalable Transfer Learning with Expert Modelscited by 48
  25. 2023Reflection-Tuning: Data Recycling Improves LLM Instruction-Tuningcited by 26
  26. 2022FAMIE: A Fast Active Learning Framework for Multilingual Information Extractioncited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 38; 2025: 57; 2026: 39.

134 papers.

Sort:

By area:

  1. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  2. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
  3. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
  4. 2025Discrete Diffusion in Large Language and Multimodal Models: A Surveycited by 61
  5. 2024Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuningcited by 57
  6. 2025Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Modelscited by 49
  7. 2025The Best Instruction-Tuning Data are Those That Fitcited by 48
  8. 2024Entropy Law: The Story Behind Data Compression and LLM Performancecited by 42
  9. 2024Quantifying the Gain in Weak-to-Strong Generalizationcited by 40
  10. 2025Large-Scale Data Selection for Instruction Tuningcited by 34
  11. 2024Can LLMs Speak For Diverse People? Tuning LLMs via Debate to Generate Controllable Controversial Statementscited by 33
  12. 2025Unhackable Temporal Rewarding for Scalable Video MLLMscited by 33
38 more
  1. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 31
  2. 2024Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Modelscited by 29
  3. 2025ATLaS: Agent Tuning via Learning Critical Stepscited by 29
  4. 2024TAGCOS: Task-agnostic Gradient Clustered Coreset Selection for Instruction Tuning Datacited by 28
  5. 2024I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigmcited by 27
  6. 2024FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Modelscited by 26
  7. 2024Super(ficial)-alignment: Strong Models May Deceive Weak Models in Weak-to-Strong Generalizationcited by 23
  8. 2025SAE-V: Interpreting Multimodal Models for Enhanced Alignmentcited by 22
  9. 2025D3: Diversity, Difficulty, and Dependability-Aware Data Selection for Sample-Efficient LLM Instruction Tuningcited by 21
  10. 2024Your Weak LLM is Secretly a Strong Teacher for Alignmentcited by 20
  11. 2024MACPO: Weak-to-Strong Alignment via Multi-Agent Contrastive Preference Optimizationcited by 20
  12. 2025Hierarchical Fine-grained Preference Optimization for Physically Plausible Video Generationcited by 20
  13. 2024Code Less, Align More: Efficient LLM Fine-tuning for Code Generation with Data Pruningcited by 19
  14. 2024Stronger Models are NOT Stronger Teachers for Instruction Tuningcited by 16
  15. 2026ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selectioncited by 16
  16. 2025Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representationscited by 14
  17. 2025R.I.P.: Better Models by Survival of the Fittest Promptscited by 13
  18. 2025Diversity as a Reward: Fine-Tuning LLMs on a Mixture of Domain-Undetermined Datacited by 13
  19. 2025LEAD: Iterative Data Selection for Efficient LLM Instruction Tuningcited by 13
  20. 2024Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Modelscited by 12
  21. 2024Chasing Random: Instruction Selection Strategies Fail to Generalizecited by 12
  22. 2025From Perceptions to Decisions: Wildfire Evacuation Decision Prediction with Behavioral Theory-informed LLMscited by 12
  23. 2025MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Spacecited by 12
  24. 2025GainRAG: Preference Alignment in Retrieval-Augmented Generation through Gain Signal Synthesiscited by 12
  25. 2025T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuningcited by 12
  26. 2025BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuningcited by 12
  27. 2024RuleR: Improving LLM Controllability by Rule-based Data Recyclingcited by 11
  28. 2024KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Servercited by 11
  29. 2025Diversity-driven Data Selection for Language Model Tuning through Sparse Autoencodercited by 11
  30. 2025InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusioncited by 11
  31. 2025Discrepancies are Virtue: Weak-to-Strong Generalization through Lens of Intrinsic Dimensioncited by 10
  32. 2024Mosaic-IT: Cost-Free Compositional Data Synthesis for Instruction Tuningcited by 9
  33. 2024Federated Data-Efficient Instruction Tuning for Large Language Modelscited by 9
  34. 2024ConTrans: Weak-to-Strong Alignment Engineering via Concept Transplantationcited by 8
  35. 2026Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetrycited by 8
  36. 2025InfiFPO: Implicit Model Fusion via Preference Optimization in Large Language Modelscited by 7
  37. 2025Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoningcited by 7
  38. 2025Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learningcited by 7

50 most cited of 134 All 134 (JSON)