# ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection

arXiv: https://arxiv.org/abs/2601.09195 · HTML: https://mlsanity.com/paper/2601.09195/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 14 Jan 2026.

## Builds on (most cited first, 30 of 32)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems](https://mlsanity.com/paper/2402.14008/) (2024)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [The False Promise of Imitating Proprietary LLMs](https://mlsanity.com/paper/2305.15717/) (2023)
- [Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning](https://mlsanity.com/paper/2402.00530/) (2024)
- [On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification](https://mlsanity.com/paper/2508.05629/) (2025)
- [A Survey on Data Selection for LLM Instruction Tuning](https://mlsanity.com/paper/2402.05123/) (2024)
- [Learn to Reason Efficiently with Adaptive Length-based Reward Shaping](https://mlsanity.com/paper/2505.15612/) (2025)
- [Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality](https://mlsanity.com/paper/2505.18227/) (2025)
- [Infinity Instruct: Scaling Instruction Selection and Synthesis to Enhance Language Models](https://mlsanity.com/paper/2506.11116/) (2025)
- [TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights](https://mlsanity.com/paper/2410.04350/) (2024)
- [Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning](https://mlsanity.com/paper/2502.01968/) (2025)
- [Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generation](https://mlsanity.com/paper/2512.06690/) (2025)
- [Low-probability Tokens Sustain Exploration in Reinforcement Learning with Verifiable Reward](https://mlsanity.com/paper/2510.03222/) (2025)
- [Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning](https://mlsanity.com/paper/2510.10974/) (2025)
- [Beyond Log Likelihood: Probability-Based Objectives for Supervised Fine-Tuning across the Model Capability Continuum](https://mlsanity.com/paper/2510.00526/) (2025)
- [Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting](https://mlsanity.com/paper/2601.02151/) (2026)
- [Shadow-FT: Tuning Instruct Model via Training on Paired Base Model](https://mlsanity.com/paper/2505.12716/) (2025)

## Built on by (most cited first, 18 of 18)

- [Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Training](https://mlsanity.com/paper/2602.12222/) (2026)
- [PERM: Psychology-grounded Empathetic Reward Modeling for Large Language Models](https://mlsanity.com/paper/2601.10532/) (2026)
- [When RL Fails after SFT: Rejuvenating Model Plasticity for Robust SFT-to-RL Handoff](https://mlsanity.com/paper/2606.09932/) (2026)
- [Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives](https://mlsanity.com/paper/2602.11424/) (2026)
- [STRIDE-ED: A Strategy-Grounded Stepwise Reasoning Framework for Empathetic Dialogue Systems](https://mlsanity.com/paper/2604.07100/) (2026)
- [PriFT: Prior-Support Guided Supervised Fine-Tuning](https://mlsanity.com/paper/2606.09396/) (2026)
- [Learning Concepts, Not Tokens: Self-Supervised Semantic Alignment for Language Models](https://mlsanity.com/paper/2603.29123/) (2026)
- [Robust Reasoning via Dynamic Token Selection for Distribution-Aligned Self-Distillation](https://mlsanity.com/paper/2606.00628/) (2026)
- [AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training](https://mlsanity.com/paper/2606.01635/) (2026)
- [From Tokens to Regions: CUDA-Sensitive Instruction Tuning for GPU Kernel Generation](https://mlsanity.com/paper/2606.16231/) (2026)
- [Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority](https://mlsanity.com/paper/2602.01227/) (2026)
- [RASFT: Rollout-Adaptive Supervised Fine-Tuning for Reasoning](https://mlsanity.com/paper/2606.07006/) (2026)
- [A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design](https://mlsanity.com/paper/2606.11189/) (2026)
- [ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training](https://mlsanity.com/paper/2609.16639/) (2026)
- [Token Utility Is Selection-Conditioned: Coupled Selection of Prompt Context and Response Supervision for Instruction Tuning](https://mlsanity.com/paper/2609.22943/) (2026)
- [Rethinking Token Reweighting for SFT: Suppress, Reverse, and Extrapolate Learned Features](https://mlsanity.com/paper/2609.33463/) (2026)
- [What Should SFT Learn? Conditional Information Gain for Generalizable Post-Training](https://mlsanity.com/paper/2610.12995/) (2026)
- [Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation](https://mlsanity.com/paper/2610.17607/) (2026)

## Abstract (arXiv)

Supervised fine-tuning (SFT) is a fundamental post-training strategy to align Large Language Models (LLMs) with human intent. However, traditional SFT often ignores the one-to-many nature of language by forcing alignment with a single reference answer, leading to the model overfitting to non-core expressions. Although our empirical analysis suggests that introducing multiple reference answers can mitigate this issue, the prohibitive data and computational costs necessitate a strategic shift: prioritizing the mitigation of single-reference overfitting over the costly pursuit of answer diversity. To achieve this, we reveal the intrinsic connection between token probability and semantic importance: high-probability tokens carry the core logical framework, while low-probability tokens are mostly replaceable expressions. Based on this insight, we propose ProFit, which selectively masks low-probability tokens to prevent surface-level overfitting. Extensive experiments confirm that ProFit consistently outperforms traditional SFT baselines on general reasoning and mathematical benchmarks.