mlsanityData to 7 Oct 2026

Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning

arXiv 2502.07154 ↗First posted 11 Feb 2025cs.LG cs.AIcited by 28

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent progress in large language models (LLMs) highlights the power of scaling test-time compute to achieve strong performance on complex tasks, such as mathematical reasoning and code generation. This raises a critical question: how should model training be modified to optimize performance under a subsequent test-time compute strategy and budget? To explore this, we focus on pass@N, a simple test-time strategy that searches for a correct answer in N independent samples. We show, surprisingly, that training with cross-entropy (CE) loss can be it misaligned with pass@N in that pass@N accuracy it decreases with longer training. We explain the origins of this misalignment in terms of model overconfidence induced by CE, and experimentally verify our prediction of overconfidence as an impediment to scaling test-time compute via pass@N. Furthermore we suggest a principled, modified training loss that is better aligned to pass@N by limiting model confidence and rescuing pass@N test performance. Our algorithm demonstrates improved mathematical reasoning on MATH and MiniF2F benchmarks under several scenarios: (1) providing answers to math questions; and (2) proving theorems by searching over proof trees of varying shapes. Overall our work underscores the importance of co-designing two traditionally separate phases of LLM development: training-time protocols and test-time search and reasoning strategies.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 15 of its 52 references that are in the map. First posted on arXiv 11 Feb 2025.

Builds on

15 of 52 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug

Other references · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  7. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  8. 2025LIMO: Less is More for Reasoningcited by 488
  9. 2022Self-critiquing models for assisting human evaluatorscited by 372
  10. 2024InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoningcited by 117
  11. 2024Preserving Diversity in Supervised Fine-Tuning of Large Language Modelscited by 68
  12. 2021Scaling Scaling Laws with Board Gamescited by 62
2 more
  1. 2024A Simple Model of Inference Scaling Lawscited by 36
  2. 2023When Less is More: Investigating Data Pruning for Pretraining LLMs at Scalecited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 11; 2026: 17.

28 papers.

Sort:

By area:

  1. 2025Rethinking Reflection in Pre-Trainingcited by 43
  2. 2025Weight Ensembling Improves Reasoning in Language Modelscited by 36
  3. 2025Differential Smoothing Mitigates Sharpening and Improves LLM Reasoningcited by 19
  4. 2025Don't Pass@k: A Bayesian Framework for Large Language Model Evaluationcited by 15
  5. 2025Reasoning Path Divergence: A New Metric and Curation Strategy to Unlock LLM Diverse Thinkingcited by 10
  6. 2026F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rarecited by 10
  7. 2025Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoningcited by 8
  8. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8
  9. 2025Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scalingcited by 7
  10. 2026LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement Learningcited by 7
  11. 2026Learning Shrinks the Hard Tail: Training-Dependent Inference Scaling in a Solvable Linear Modelcited by 6
  12. 2025A Theory of Inference Compute Scaling: Reasoning through Directed Stochastic Skill Searchcited by 5
16 more
  1. 2025BrowseConf: Confidence-Guided Test-Time Scaling for Web Agentscited by 5
  2. 2025Demystifying LLM-as-a-Judge: Analytically Tractable Model for Inference-Time Scalingcited by 4
  3. 2026Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarkscited by 4
  4. 2026TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuningcited by 3
  5. 2026TailSFT: Filtered Fine-Tuning Improves Post-Training Performancecited by 2
  6. 2026Computational Arbitrage in AI Model Marketscited by 1
  7. 2026Asymptotic Universal Alignment: A New Alignment Framework via Test-Time Scalingcited by 0
  8. 2026Do Not Imitate, Reinforce: Iterative Classification via Belief Refinementcited by 0
  9. 2026IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMscited by 0
  10. 2026Automatic Hard Example Synthesis with Multi-Level Agentic Data Curationcited by 0
  11. 2026Towards Valid B-Rep Generation: Training-Free Wireframe Anomaly Detection and Repaircited by 0
  12. 2026Do Quantum AIs Dream in Paths? Path-Integral Slow Thinking through Grover Interferencecited by 0
  13. 2026Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidancecited by 0
  14. 2026Direct Optimization of Generators for Search in Automated Theorem Provingcited by 0
  15. 2026Entropy Regularization: A Free Correction to Cross-Entropy for Verified Demonstrationscited by 0
  16. 2026SoFT: Soft Targets for Generalizable LLM Fine-Tuningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Maximizing Prefix-Confidence at Test-Time Efficiently Improves Mathematical Reasoning
  2. 2025Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention
  3. 2025Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling
  4. 2025Are We Scaling the Right Thing? A System Perspective on Test-Time Scaling
  5. 2025The Art of Scaling Test-Time Compute for Large Language Models
  6. 2026When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling
  7. 2025On the Role of Temperature Sampling in Test-Time Scaling
  8. 2026Compute Aligned Training: Optimizing for Test Time Inference