mlsanityData to 14 Oct 2026

Revisiting the Test-Time Scaling of o1-like Models: Do they Truly Possess Test-Time Scaling Capabilities?

arXiv 2502.12215 ↗First posted 17 Feb 2025cs.LG cs.AI cs.CLcited by 70

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

The advent of test-time scaling in large language models (LLMs), exemplified by OpenAI's o1 series, has advanced reasoning capabilities by scaling computational resource allocation during inference. While successors like QwQ, Deepseek-R1 (R1) and LIMO replicate these advancements, whether these models truly possess test-time scaling capabilities remains underexplored. This study found that longer CoTs of these o1-like models do not consistently enhance accuracy; in fact, correct solutions are often shorter than incorrect ones for the same questions. Further investigation shows this phenomenon is closely related to models' self-revision capabilities - longer CoTs contain more self-revisions, which often lead to performance degradation. We then compare sequential and parallel scaling strategies on QwQ, R1 and LIMO, finding that parallel scaling achieves better coverage and scalability. Based on these insights, we propose Shortest Majority Vote, a method that combines parallel scaling strategies with CoT length characteristics, significantly improving models' test-time scalability compared to conventional majority voting approaches.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 39 of its 46 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

39 of 46 references.

Closest ancestors · 18

Same direction, newest first

  1. 2025s1: Simple test-time scaling31 Jan
  2. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs30 Jan
  3. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  4. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  5. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspective18 Dec
  6. 2024Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems12 Dec
  7. 2024O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?25 Nov
  8. 2024Enhancing LLM Reasoning with Reward-guided Tree Search18 Nov
  9. 2024TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling18 Oct
  10. 2024O1 Replication Journey: A Strategic Progress Report -- Part 18 Oct
  11. 2024Training Language Models to Self-Correct via Reinforcement Learning19 Sep
  12. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
6 more
  1. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoning26 Apr
  2. 2023Large Language Models Cannot Self-Correct Reasoning Yet3 Oct
  3. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Training29 Sep
  4. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing19 May
  5. 2023Self-Evaluation Guided Beam Search for Reasoning1 May
  6. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 21

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  7. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  8. 2023Teaching Large Language Models to Self-Debugcited by 988
  9. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  10. 2025LIMO: Less is More for Reasoningcited by 498
  11. 2025Training Language Models to Reason Efficientlycited by 247
  12. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
9 more
  1. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 211
  2. 2023Planning with Large Language Models for Code Generationcited by 182
  3. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 162
  4. 2024BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Samplingcited by 121
  5. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  6. 2023OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoningcited by 85
  7. 2024BOND: Aligning LLMs with Best-of-N Distillationcited by 76
  8. 2024Variational Best-of-N Alignmentcited by 51
  9. 2024When is Tree Search Useful for LLM Planning? It Depends on the Discriminatorcited by 47

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 52; 2026: 18.

70 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  3. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learningcited by 192
  4. 2025The Surprising Effectiveness of Negative Reinforcement in LLM Reasoningcited by 162
  5. 2025Context Length Alone Hurts LLM Performance Despite Perfect Retrievalcited by 127
  6. 2025Efficient Reasoning Models: A Surveycited by 95
  7. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 92
  8. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 48
  9. 2025Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoningcited by 34
  10. 2025Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspectivecited by 34
  11. 2026How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Taskscited by 33
  12. 2025Learning to Reason Across Parallel Samples for LLM Reasoningcited by 32
38 more
  1. 2025Prover Agent: An Agent-Based Framework for Formal Mathematical Proofscited by 31
  2. 2025DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Modelscited by 30
  3. 2025AlphaOne: Reasoning Models Thinking Slow and Fast at Test Timecited by 29
  4. 2025Heimdall: test-time scaling on the generative verificationcited by 26
  5. 2025When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoningcited by 26
  6. 2025OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critiquecited by 25
  7. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  8. 2025Learning from Peers in Reasoning Modelscited by 16
  9. 2025AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Taskscited by 16
  10. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  11. 2025On Reasoning Strength Planning in Large Reasoning Modelscited by 13
  12. 2025Compile Scene Graphs with Reinforcement Learningcited by 11
  13. 2025The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched Computecited by 11
  14. 2025RoT: Enhancing Table Reasoning with Iterative Row-Wise Traversalscited by 10
  15. 2025Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Frameworkcited by 10
  16. 2025SolverLLM: Leveraging Test-Time Scaling for Optimization Problem via LLM-Guided Searchcited by 10
  17. 2025DiRL: An Efficient Post-Training Framework for Diffusion Language Modelscited by 10
  18. 2025Diversity-Enhanced Reasoning for Subjective Questionscited by 9
  19. 2026VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Modelcited by 9
  20. 2025ThinkingViT: Matryoshka Thinking Vision Transformer for Elastic Inferencecited by 7
  21. 2026Unified Generation and Self-Verification for Vision-Language Models via Advantage Decoupled Preference Optimizationcited by 7
  22. 2025Knowledge Augmented Complex Problem Solving with Large Language Models: A Surveycited by 6
  23. 2025T2: An Adaptive Test-Time Scaling Strategy for Contextual Question Answeringcited by 6
  24. 2025Reasoning LLMs are Wandering Solution Explorerscited by 6
  25. 2025Reasoning or Not? A Comprehensive Evaluation of Reasoning LLMs for Dialogue Summarizationcited by 6
  26. 2025Synthesizing Visual Concepts as Vision-Language Programscited by 6
  27. 2025Think in Parallel, Answer as One: Logit Averaging for Open-Ended Reasoningcited by 6
  28. 2025Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decodingcited by 5
  29. 2025RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Priorcited by 5
  30. 2025Uncertainty Profiles for LLMs: Uncertainty Source Decomposition and Adaptive Model-Metric Selectioncited by 4
  31. 2025Test-Time Scaling with Reflective Generative Modelcited by 4
  32. 2025Generalizing Test-time Compute-optimal Scaling as an Optimizable Graphcited by 4
  33. 2026Language of Thought Shapes Output Diversity in Large Language Modelscited by 4
  34. 2026Internalizing LLM Reasoning via Discovery and Replay of Latent Actionscited by 4
  35. 2025Rethinking Test-Time Scaling for Medical AI: Model and Task-Aware Strategies for LLMs and VLMscited by 3
  36. 2025Pruning Long Chain-of-Thought of Large Reasoning Models via Small-Scale Preference Optimizationcited by 3
  37. 2025ATTS: Asynchronous Test-Time Scaling via Conformal Predictioncited by 3
  38. 2025Mitigating Strategy-Selection Bias in Reasoning for More Effective Test-Time Scalingcited by 3

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Understanding the Role of Training Data in Test-Time Scaling
  2. 2025Are We Scaling the Right Thing? A System Perspective on Test-Time Scaling
  3. 2025The Art of Scaling Test-Time Compute for Large Language Models
  4. 2026Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility
  5. 2025Test-time Scaling of LLMs: A Survey from A Subproblem Structure Perspective
  6. 2025Scaling Test-time Compute for LLM Agents
  7. 2025Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signals
  8. 2026On Test-Time Scaling for Vision-Language Models