mlsanityData to 7 Oct 2026

Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters

arXiv 2408.03314 ↗First posted 6 Aug 2024cs.LG cs.CLcited by 1,928

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Enabling LLMs to improve their outputs by using more test-time computation is a critical step towards building generally self-improving agents that can operate on open-ended natural language. In this paper, we study the scaling of inference-time computation in LLMs, with a focus on answering the question: if an LLM is allowed to use a fixed but non-trivial amount of inference-time compute, how much can it improve its performance on a challenging prompt? Answering this question has implications not only on the achievable performance of LLMs, but also on the future of LLM pretraining and how one should tradeoff inference-time and pre-training compute. Despite its importance, little research attempted to understand the scaling behaviors of various test-time inference methods. Moreover, current work largely provides negative results for a number of these strategies. In this work, we analyze two primary mechanisms to scale test-time computation: (1) searching against dense, process-based verifier reward models; and (2) updating the model's distribution over a response adaptively, given the prompt at test time. We find that in both cases, the effectiveness of different approaches to scaling test-time compute critically varies depending on the difficulty of the prompt. This observation motivates applying a "compute-optimal" scaling strategy, which acts to most effectively allocate test-time compute adaptively per prompt. Using this compute-optimal strategy, we can improve the efficiency of test-time compute scaling by more than 4x compared to a best-of-N baseline. Additionally, in a FLOPs-matched evaluation, we find that on problems where a smaller base model attains somewhat non-trivial success rates, test-time compute can be used to outperform a 14x larger model.

Key facts. Cited by 1,928 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 21 of its 32 references that are in the map. First posted on arXiv 6 Aug 2024.

Builds on

21 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Think before you speak: Training Language Models With Pause Tokens3 Oct
  2. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training29 Sep
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May

Other references · 18

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  5. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  6. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  7. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  8. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  9. 2024Tool Learning with Large Language Models: A Surveycited by 221
  10. 2023Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Lawscited by 155
  11. 2024LLM Critics Help Catch LLM Bugscited by 138
  12. 2023Hypothesis Search: Inductive Reasoning with Language Modelscited by 119
6 more
  1. 2023Can Large Language Models Really Improve by Self-critiquing Their Own Plans?cited by 107
  2. 2024RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Foldcited by 99
  3. 2023GPT-4 Doesn't Know It's Wrong: An Analysis of Iterative Prompting for Reasoning Problemscited by 88
  4. 2021Scaling Scaling Laws with Board Gamescited by 62
  5. 2024Does your data spark joy? Performance gains from domain upsampling at the end of trainingcited by 51
  6. 2024A Critical Evaluation of AI Feedback for Aligning Large Language Modelscited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 146; 2025: 993; 2026: 789.

1,928 papers.

Sort:

By area:

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  4. 2025s1: Simple test-time scalingcited by 1,311
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  6. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
  7. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  8. 2025Dream 7B: Diffusion Large Language Modelscited by 515
  9. 2025LIMO: Less is More for Reasoningcited by 488
  10. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  11. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  12. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
38 more
  1. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  2. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 381
  3. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  4. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 377
  5. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  6. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 308
  7. 2024Latent Action Pretraining from Videoscited by 302
  8. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  9. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruningcited by 278
  10. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  11. 2024Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoningcited by 267
  12. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 261
  13. 2025TTRL: Test-Time Reinforcement Learningcited by 253
  14. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 226
  15. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 225
  16. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  17. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  18. 2025Video models are zero-shot learners and reasonerscited by 208
  19. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
  20. 2025Deep Think with Confidencecited by 199
  21. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 187
  22. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
  23. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certaintycited by 182
  24. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  25. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  26. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  27. 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 168
  28. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  29. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoningcited by 160
  30. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 158
  31. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  32. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 154
  33. 2025ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learningcited by 152
  34. 2025Do generative video models understand physical principles?cited by 151
  35. 2026Entropy-Aware On-Policy Distillation of Language Modelscited by 151
  36. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  37. 2024O1 Replication Journey: A Strategic Progress Report -- Part 1cited by 140
  38. 2025Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correctioncited by 140

50 most cited of 1,928 All 1,928 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory
  2. 2024Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models
  3. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
  4. 2026ThinkBooster: A Unified Framework for Seamless Test-Time Scaling of LLM Reasoning
  5. 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking
  6. 2025Test-time Scaling of LLMs: A Survey from A Subproblem Structure Perspective
  7. 2025Demystifying LLM-as-a-Judge: Analytically Tractable Model for Inference-Time Scaling
  8. 2025Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentation