mlsanityData to 14 Oct 2026

Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time Scaling

arXiv 2502.06703 ↗First posted 10 Feb 2025cs.CLcited by 136

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Test-Time Scaling (TTS) is an important method for improving the performance of Large Language Models (LLMs) by using additional computation during the inference phase. However, current studies do not systematically analyze how policy models, Process Reward Models (PRMs), and problem difficulty influence TTS. This lack of analysis limits the understanding and practical use of TTS methods. In this paper, we focus on two core questions: (1) What is the optimal approach to scale test-time computation across different policy models, PRMs, and problem difficulty levels? (2) To what extent can extended computation improve the performance of LLMs on complex tasks, and can smaller language models outperform larger ones through this approach? Through comprehensive experiments on MATH-500 and challenging AIME24 tasks, we have the following observations: (1) The compute-optimal TTS strategy is highly dependent on the choice of policy model, PRM, and problem difficulty. (2) With our compute-optimal TTS strategy, extremely small policy models can outperform larger models. For example, a 1B LLM can exceed a 405B LLM on MATH-500. Moreover, on both MATH-500 and AIME24, a 0.5B LLM outperforms GPT-4o, a 3B LLM surpasses a 405B LLM, and a 7B LLM beats o1 and DeepSeek-R1, while with higher inference efficiency. These findings show the significance of adapting TTS strategies to the specific characteristics of each task and model and indicate that TTS is a promising approach for enhancing the reasoning abilities of LLMs.

Key facts. Cited by 136 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 40 of its 77 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

40 of 77 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?25 Nov
  2. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions21 Nov
  3. 2024O1 Replication Journey: A Strategic Progress Report -- Part 18 Oct
  4. 2024Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation3 Oct
  5. 2024Training Language Models to Self-Correct via Reinforcement Learning19 Sep
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  7. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models1 Aug
  8. 2024V-STaR: Training Verifiers for Self-Taught Reasoners9 Feb

Other references · 32

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2024Qwen2.5 Technical Reportcited by 4,942
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2023Mistral 7Bcited by 3,164
  9. 2024Qwen2 Technical Reportcited by 2,120
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  11. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  12. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
20 more
  1. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  2. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 625
  3. 2025Process Reinforcement through Implicit Rewardscited by 504
  4. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  5. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  6. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 415
  7. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  8. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  9. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 279
  10. 2024Free Process Rewards without Process Labelscited by 165
  11. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  12. 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Modelscited by 99
  13. 2024OpenR: An Open Source Framework for Advanced Reasoning with Large Language Modelscited by 94
  14. 2024Process Reward Model with Q-Value Rankingscited by 80
  15. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Searchcited by 53
  16. 2024MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Timecited by 41
  17. 2024Entropy-Regularized Process Reward Modelcited by 38
  18. 2024Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes Oncited by 24
  19. 2024Scaling Inference-Time Search with Vision Value Model for Improved Visual Comprehensioncited by 23
  20. 2024CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoningcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 101; 2026: 35.

136 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  3. 2025TTRL: Test-Time Reinforcement Learningcited by 275
  4. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  5. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  6. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 124
  7. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  8. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoningcited by 97
  9. 2025Efficient Reasoning Models: A Surveycited by 95
  10. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 73
  11. 2025Scaling Test-time Compute for LLM Agentscited by 63
  12. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Timecited by 48
38 more
  1. 2025The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Managementcited by 48
  2. 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMscited by 46
  3. 2025Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Spacecited by 43
  4. 2025Video-T1: Test-Time Scaling for Video Generationcited by 40
  5. 2025Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVRcited by 39
  6. 2025ETTRL: Balancing Exploration and Exploitation in LLM Test-Time Reinforcement Learning Via Entropy Mechanismcited by 36
  7. 2025Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language Modelscited by 33
  8. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  9. 2025Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMscited by 29
  10. 2025Two Heads are Better Than One: Test-time Scaling of Multi-agent Collaborative Reasoningcited by 29
  11. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
  12. 2025Shrinking the Generation-Verification Gap with Weak Verifierscited by 24
  13. 2025Entropy-Gated Branching for Efficient Test-Time Reasoningcited by 22
  14. 2025Know What You Don't Know: Uncertainty Calibration of Process Reward Modelscited by 21
  15. 2025Do We Truly Need So Many Samples? Multi-LLM Repeated Sampling Efficiently Scales Test-Time Computecited by 20
  16. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 19
  17. 2025Variation in Verification: Understanding Verification Dynamics in Large Language Modelscited by 19
  18. 2025RoVer: Robot Reward Model as Test-Time Verifier for Vision-Language-Action Modelcited by 19
  19. 2025Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signalscited by 18
  20. 2025SLOT: Sample-specific Language Model Optimization at Test-timecited by 18
  21. 2025Let's (not) just put things in Context: Test-Time Training for Long-Context LLMscited by 18
  22. 2025ViLBench: A Suite for Vision-Language Process Reward Modelingcited by 16
  23. 2025AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Taskscited by 16
  24. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  25. 2025Every Rollout Counts: Optimal Resource Allocation for Efficient Test-Time Scalingcited by 15
  26. 2026TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokenscited by 15
  27. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 14
  28. 2025SSRL: Self-Search Reinforcement Learningcited by 14
  29. 2025T1: Tool-integrated Verification for Test-time Compute Scaling in Small Language Modelscited by 13
  30. 2025Is PRM Necessary? Problem-Solving RL Implicitly Induces PRM Capability in LLMscited by 13
  31. 2025Value-Guided Search for Efficient Chain-of-Thought Reasoningcited by 13
  32. 2025Let Me Think! A Long Chain-of-Thought Can Be Worth Exponentially Many Short Onescited by 13
  33. 2025CoSteer: Collaborative Decoding-Time Personalization via Local Delta Steeringcited by 13
  34. 2025Bottom-up Domain-specific Superintelligence: A Reliable Knowledge Graph is What We Needcited by 13
  35. 2025Energy Use of AI Inference, Efficiency Pathways, and Test-Time Scalingcited by 13
  36. 2025Attention as a Compass: Efficient Exploration for Process-Supervised RL in Reasoning Modelscited by 13
  37. 2025SPINE: Token-Selective Test-Time Reinforcement Learning with Entropy-Band Regularizationcited by 12
  38. 2025Scaling LLM Test-Time Compute with Mobile NPU on Smartphonescited by 11

50 most cited of 136 All 136 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025The Art of Scaling Test-Time Compute for Large Language Models
  2. 2025Test-time Scaling of LLMs: A Survey from A Subproblem Structure Perspective
  3. 2025Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory
  4. 2025Are We Scaling the Right Thing? A System Perspective on Test-Time Scaling
  5. 2025Mitigating Strategy-Selection Bias in Reasoning for More Effective Test-Time Scaling
  6. 2025The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Compute
  7. 2026ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning
  8. 2026Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling