mlsanityData to 14 Oct 2026

Rethinking Data Selection for Supervised Fine-Tuning

arXiv 2402.06094 ↗First posted 8 Feb 2024cs.CLcited by 35

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Although supervised finetuning (SFT) has emerged as an essential technique to align large language models with humans, it is considered superficial, with style learning being its nature. At the same time, recent works indicate the importance of data selection for SFT, showing that finetuning with high-quality and diverse subsets of the original dataset leads to superior downstream performance. In this work, we rethink the intuition behind data selection for SFT. Considering SFT is superficial, we propose that essential demonstrations for SFT should focus on reflecting human-like interactions instead of data quality or diversity. However, it is not straightforward to directly assess to what extent a demonstration reflects human styles. Towards an initial attempt in this direction, we find selecting instances with long responses is surprisingly more effective for SFT than utilizing full datasets or instances selected based on quality and diversity. We hypothesize that such a simple heuristic implicitly mimics a crucial aspect of human-style conversation: detailed responses are usually more helpful.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 1 of its 74 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

1 of 74 references.

References · 1

  1. 2023AlpaGasus: Training A Better Alpaca with Fewer Datacited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 19; 2026: 6.

35 papers.

Sort:

By area:

  1. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
  2. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  3. 2025Intern-S1: A Scientific Multimodal Foundation Modelcited by 53
  4. 2025Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Modelscited by 51
  5. 2024Entropy Law: The Story Behind Data Compression and LLM Performancecited by 44
  6. 2024Learning or Self-aligning? Rethinking Instruction Fine-tuningcited by 43
  7. 2024Safety-Aware Fine-Tuning of Large Language Modelscited by 34
  8. 2024Rethinking Data Selection at Scale: Random Selection is Almost All You Needcited by 22
  9. 2024The Open Source Advantage in Large Language Models (LLMs)cited by 18
  10. 2025Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinementcited by 18
  11. 2025Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Searchcited by 16
  12. 2024LLM Roleplay: Simulating Human-Chatbot Interactioncited by 15
23 more
  1. 2025Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representationscited by 15
  2. 2024Chasing Random: Instruction Selection Strategies Fail to Generalizecited by 13
  3. 2025RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Modelscited by 13
  4. 2025IDEAL: Data Equilibrium Adaptation for Multi-Capability Language Model Alignmentcited by 8
  5. 2025DeepSound-V1: Start to Think Step-by-Step in the Audio Generation from Videoscited by 4
  6. 2026Data Selection for Multi-turn Dialogue Instruction Tuningcited by 4
  7. 2025MergeIT: From Selection to Merging for Efficient Instruction Tuningcited by 3
  8. 2025TAG-INSTRUCT: Controlled Instruction Complexity Enhancement through Structure-based Augmentationcited by 3
  9. 2025CircuitSeer: Mining High-Quality Data by Probing Mathematical Reasoning Circuits in LLMscited by 3
  10. 2025CLLoRA: An Approach to Measure the Effects of the Context Length for LLM Fine-Tuningcited by 2
  11. 2025Exploring Solution Divergence and Its Effect on Large Language Model Problem Solvingcited by 2
  12. 2025Computational Budget Should Be Considered in Data Selectioncited by 2
  13. 2024Mistral-C2F: Coarse to Fine Actor for Analytical and Reasoning Enhancement in RLHF and Effective-Merged LLMscited by 1
  14. 2024Boosting LLM via Learning from Data Iteratively and Selectivelycited by 1
  15. 2025Beyond QA Pairs: Assessing Parameter-Efficient Fine-Tuning for Fact Embedding in LLMscited by 1
  16. 2025V-VAE: A Variational Auto Encoding Framework Towards Fine-Grained Control over Human-Like Chatcited by 1
  17. 2025bi-GRPO: Bidirectional Optimization for Jailbreak Backdoor Injection on LLMscited by 1
  18. 2026Short Chains, Deep Thoughts: Balancing Reasoning Efficiency and Intra-Segment Capability via Split-Merge Optimizationcited by 1
  19. 2026Self-Play Enhancement via Advantage-Weighted Refinement in Online Federated LLM Fine-Tuning with Real-Time Feedbackcited by 1
  20. 2026RIZZ: Routing Interactions to Near Zero-Interference Zones for Continual Adaptation of Black-Box Agentscited by 1
  21. 2025Motif 2.6B Technical Reportcited by 0
  22. 2026A Context Engineering Framework for Improving Enterprise AI Agents based on Digital-Twin MDPcited by 0
  23. 2026AlignDiff: Exploiting Model-Intrinsic Information for Better Preference Data Selectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models
  2. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models
  3. 2026Diversity in Large Language Models under Supervised Fine-Tuning
  4. 2025Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
  5. 2025Efficient Response Generation Strategy Selection for Fine-Tuning Large Language Models Through Self-Aligned Perplexity
  6. 2024ToolFlow: Boosting LLM Tool-Calling Through Natural and Coherent Dialogue Synthesis
  7. 2026The Significance of Style Diversity in Annotation-Free Synthetic Data Generation
  8. 2025Balancing the Budget: Understanding Trade-offs Between Supervised and Preference-Based Finetuning