# Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning

arXiv: https://arxiv.org/abs/2510.10974 · HTML: https://mlsanity.com/paper/2510.10974/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 12 of its 50 references that are in the map.
- First posted on arXiv 13 Oct 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 12 of 12)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning](https://mlsanity.com/paper/2506.01939/) (2025)
- [2 OLMo 2 Furious](https://mlsanity.com/paper/2501.00656/) (2024)
- [DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning](https://mlsanity.com/paper/2504.11456/) (2025)
- [Evaluating the Performance of Large Language Models on GAOKAO Benchmark](https://mlsanity.com/paper/2305.12474/) (2023)
- [MathScale: Scaling Instruction Tuning for Mathematical Reasoning](https://mlsanity.com/paper/2403.02884/) (2024)
- [On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification](https://mlsanity.com/paper/2508.05629/) (2025)
- [Unveiling Over-Memorization in Finetuning LLMs for Reasoning Tasks](https://mlsanity.com/paper/2508.04117/) (2025)

## Built on by (most cited first, 20 of 20)

- [Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning](https://mlsanity.com/paper/2601.07408/) (2026)
- [ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection](https://mlsanity.com/paper/2601.09195/) (2026)
- [Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structure](https://mlsanity.com/paper/2602.08783/) (2026)
- [SPPO: Sequence-Level PPO for Long-Horizon Reasoning Tasks](https://mlsanity.com/paper/2604.08865/) (2026)
- [RelayLLM: Efficient Reasoning via Collaborative Decoding](https://mlsanity.com/paper/2601.05167/) (2026)
- [SED-SFT: Selectively Encouraging Diversity in Supervised Fine-Tuning](https://mlsanity.com/paper/2602.07464/) (2026)
- [Beyond Uniform Credit: Causal Credit Assignment for Policy Optimization](https://mlsanity.com/paper/2602.09331/) (2026)
- [Decoupling Skeleton and Flesh: Efficient Multimodal Table Reasoning with Disentangled Alignment and Structure-aware Guidance](https://mlsanity.com/paper/2602.03491/) (2026)
- [Gradients Must Earn Their Influence: Unifying SFT with Generalized Entropic Objectives](https://mlsanity.com/paper/2602.11424/) (2026)
- [GIFT: Reconciling Post-Training Objectives via Variational Finite-Temperature Gibbs Initialization](https://mlsanity.com/paper/2601.09233/) (2026)
- [SAT: Balancing Reasoning Accuracy and Efficiency with Stepwise Adaptive Thinking](https://mlsanity.com/paper/2604.07922/) (2026)
- [TS-PEFT: Unveiling Token-Level Redundancy in Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2511.16147/) (2025)
- [GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models](https://mlsanity.com/paper/2605.01256/) (2026)
- [Re-feeding Is Not Replaying: Measuring Replay Noise in Counterfactual Token-Credit Estimation](https://mlsanity.com/paper/2606.15621/) (2026)
- [Beyond Entropy: Learning from Token-Level Distributional Deviations for LLM Reasoning](https://mlsanity.com/paper/2606.19771/) (2026)
- [Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority](https://mlsanity.com/paper/2602.01227/) (2026)
- [A Unifying Lens on Supervised Fine-Tuning Through Target Distribution Design](https://mlsanity.com/paper/2606.11189/) (2026)
- [Which Tokens Should SFT Actually Learn? A Token-Trimming Perspective on Mathematical Reasoning](https://mlsanity.com/paper/2609.09707/) (2026)
- [Training LLM Judges from Language Feedback via Position-Selective Self-Distillation](https://mlsanity.com/paper/2609.38792/) (2026)
- [Semifactual Credit-Augmented Policy Optimization](https://mlsanity.com/paper/2609.40360/) (2026)

## Abstract (arXiv)

Large language models (LLMs) primarily rely on supervised fine-tuning (SFT) as a key method to adapt pre-trained models to domain-specific tasks such as mathematical reasoning. However, standard SFT uniformly penalizes all tokens, neglecting that only a small subset of critical tokens determines reasoning correctness. This uniform supervision often causes reduced output diversity and limited generalization. We propose Critical Token Fine-tuning (CFT), a simple yet effective approach that updates only tokens identified as functionally indispensable via counterfactual perturbations. By focusing gradient signals on these decisive reasoning steps while preserving the diversity of non-critical tokens, CFT can enhance both generation and diversity. Extensive experiments on five models across three families (Qwen, OLMo, LLaMA) and eleven mathematical reasoning benchmarks show that CFT, despite fine-tuning on less than 12% of tokens, consistently outperforms standard SFT. Moreover, CFT enables test-time scaling through improved sampling diversity and provides a stronger initialization for reinforcement learning, sustaining performance gains in later training stages while maintaining higher entropy for better exploration. These results highlight CFT as a practical and general framework for efficient and robust LLM fine-tuning.