mlsanityData to 14 Oct 2026

ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection

arXiv 2601.09195 ↗First posted 14 Jan 2026cs.CL cs.AIcited by 18

Not on the map. Why?

Abstract (arXiv)

Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many nature of language by forcing alignment with a single reference answer, leading to the model overfitting to non-core expressions. Although our empirical analysis suggests that introducing multiple reference answers can mitigate this issue, the prohibitive data and computational costs necessitate a strategic shift: prioritizing the mitigation of single-reference overfitting over the costly pursuit of answer diversity. To achieve this, we reveal the intrinsic connection between token probability and semantic importance: high-probability tokens carry the core logical framework, while low-probability tokens are mostly replaceable expressions. Based on this insight, we propose ProFit, which selectively masks low-probability tokens to prevent surface-level overfitting. Extensive experiments confirm that ProFit consistently outperforms traditional SFT baselines on general reasoning and mathematical benchmarks.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Builds on 32 of its 45 references that are in the map. First posted on arXiv 14 Jan 2026.

Builds on

32 of 45 references.

References · 32

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  8. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  9. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  10. 2025s1: Simple test-time scalingcited by 1,341
  11. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  12. 2023LIMA: Less Is More for Alignmentcited by 1,100
20 more
  1. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  2. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  3. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  4. 2023The False Promise of Imitating Proprietary LLMscited by 263
  5. 2024Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuningcited by 138
  6. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 127
  7. 2024A Survey on Data Selection for LLM Instruction Tuningcited by 77
  8. 2025Learn to Reason Efficiently with Adaptive Length-based Reward Shapingcited by 75
  9. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 50
  10. 2025Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Modelscited by 50
  11. 2024TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weightscited by 42
  12. 2025Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuningcited by 39
  13. 2025Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generationcited by 30
  14. 2025Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Rewardcited by 25
  15. 2025Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuningcited by 20
  16. 2025Beyond Log Likelihood: Probability-Based Objectives for Supervised Fine-Tuning across the Model Capability Continuumcited by 18
  17. 2026Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgettingcited by 16
  18. 2025Shadow-FT: Tuning Instruct Model via Training on Paired Base Modelcited by 10
  19. 2025Importance-Aware Data Selection for Efficient LLM Instruction Tuningcited by 7
  20. 2025Training LLMs Beyond Next Token Prediction - Filling the Mutual Information Gapcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 18.

18 papers.

Sort:

By area:

  1. 2026Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Trainingcited by 11
  2. 2026PERM: Psychology-grounded Empathetic Reward Modeling for Large Language Modelscited by 5
  3. 2026When RL Fails after SFT: Rejuvenating Model Plasticity for Robust SFT-to-RL Handoffcited by 4
  4. 2026Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectivescited by 3
  5. 2026STRIDE-ED: A Strategy-Grounded Stepwise Reasoning Framework for Empathetic Dialogue Systemscited by 3
  6. 2026PriFT: Prior-Support Guided Supervised Fine-Tuningcited by 2
  7. 2026Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Modelscited by 1
  8. 2026Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillationcited by 1
  9. 2026AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Trainingcited by 1
  10. 2026From Tokens to Regions: CUDA-Sensitive Instruction Tuning for GPU Kernel Generationcited by 1
  11. 2026Supervised Fine-Tuning Needs to Unlock the Potential of Token Prioritycited by 0
  12. 2026RASFT: Rollout-Adaptive Supervised Fine-Tuning for Reasoningcited by 0
6 more
  1. 2026A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Designcited by 0
  2. 2026ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Trainingcited by 0
  3. 2026Token Utility Is Selection-Conditioned: Coupled Selection of Prompt Context and Response Supervision for Instruction Tuningcited by 0
  4. 2026Rethinking Token Reweighting for SFT: Suppress, Reverse, and Extrapolate Learned Featurescited by 0
  5. 2026What Should SFT Learn? Conditional Information Gain for Generalizable Post-Trainingcited by 0
  6. 2026Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillationcited by 0