mlsanityData to 7 Oct 2026

Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority

arXiv 2602.01227 ↗First posted 1 Feb 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The transition from fitting empirical data to achieving true human utility is fundamentally constrained by a granularity mismatch, where fine-grained autoregressive generation is often supervised by coarse or uniform signals. This position paper advocates Token Priority as the essential bridge, formalizing Supervised Fine-Tuning (SFT) not as simple optimization but as a precise distribution reshaping process that aligns raw data with the ideal alignment manifold. We analyze recent breakthroughs through this unified lens, categorizing them into two distinct regimes: Positive Priority for noise filtration and Signed Priority for toxic modes unlearning. We revisit existing progress and limitations, identify key challenges, and suggest directions for future research.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 71 of its 83 references that are in the map. First posted on arXiv 1 Feb 2026.

Builds on

71 of 83 references.

References · 71

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2022Emergent Abilities of Large Language Modelscited by 2,483
  7. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  8. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  9. 2019How Can We Know What Language Models Know?cited by 953
  10. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  11. 2023Textbooks Are All You Needcited by 561
  12. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
38 more
  1. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  2. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 343
  3. 2020Gradient Starvation: A Learning Proclivity in Neural Networkscited by 262
  4. 2023Inverse Scaling: When Bigger Isn't Bettercited by 205
  5. 2024Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Lawscited by 137
  6. 2024Rho-1: Not All Tokens Are What You Needcited by 132
  7. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  8. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
  9. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 106
  10. 2023A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directionscited by 101
  11. 2024Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuningcited by 90
  12. 2024Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuningcited by 85
  13. 2024Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capabilitycited by 80
  14. 2025SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoningcited by 80
  15. 2022Inverse scaling can become U-shapedcited by 76
  16. 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learningcited by 74
  17. 2024Preserving Diversity in Supervised Fine-Tuning of Large Language Modelscited by 68
  18. 2023Instruction Mining: Instruction Data Selection for Tuning Large Language Modelscited by 57
  19. 2024TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weightscited by 42
  20. 2025Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuningcited by 39
  21. 2024SelectIT: Selective Instruction Tuning for LLMs via Uncertainty-Aware Self-Reflectioncited by 34
  22. 2025Steering LLM Thinking with Budget Guidancecited by 30
  23. 2025Fine-tuning Large Language Models for Improving Factuality in Legal Question Answeringcited by 28
  24. 2024Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuningcited by 27
  25. 2025Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Rewardcited by 25
  26. 2023Explanation-based Finetuning Makes Models More Robust to Spurious Cuescited by 24
  27. 2024Mitigating Large Language Model Hallucination with Faithful Finetuningcited by 24
  28. 2024RobustFT: Robust Supervised Fine-tuning for Large Language Models under Noisy Responsecited by 24
  29. 2025Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuningcited by 20
  30. 2021A Few More Examples May Be Worth Billions of Parameterscited by 19
  31. 2025Revisiting LLM Reasoning via Information Bottleneckcited by 19
  32. 2024RECOST: External Knowledge Guided Data-efficient Instruction Tuningcited by 18
  33. 2025SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMscited by 17
  34. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  35. 2026ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selectioncited by 16
  36. 2023Aligning Language Models with Offline Learning from Human Feedbackcited by 14
  37. 2025Token Weighting for Long-Range Language Modelingcited by 14
  38. 2026Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgettingcited by 14

50 most cited of 71 All 71 (JSON)

Built on by

Not cited yet.