# Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking

arXiv: https://arxiv.org/abs/2503.19855 · HTML: https://mlsanity.com/paper/2503.19855/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 16 of its 22 references that are in the map.
- First posted on arXiv 25 Mar 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 16 of 16)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models](https://mlsanity.com/paper/2310.04406/) (2023)
- [Search-o1: Agentic Search-Enhanced Large Reasoning Models](https://mlsanity.com/paper/2501.05366/) (2025)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models](https://mlsanity.com/paper/2408.00724/) (2024)
- [Active Prompting with Chain-of-Thought for Large Language Models](https://mlsanity.com/paper/2302.12246/) (2023)
- [O1 Replication Journey: A Strategic Progress Report -- Part 1](https://mlsanity.com/paper/2410.18982/) (2024)
- [Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning](https://mlsanity.com/paper/2502.18080/) (2025)
- [KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection](https://mlsanity.com/paper/2310.09044/) (2023)
- [A Simple Model of Inference Scaling Laws](https://mlsanity.com/paper/2410.16377/) (2024)

## Built on by (most cited first, 30 of 46)

- [Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time](https://mlsanity.com/paper/2504.12329/) (2025)
- [EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity](https://mlsanity.com/paper/2507.21848/) (2025)
- [Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models](https://mlsanity.com/paper/2505.03469/) (2025)
- [SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding](https://mlsanity.com/paper/2511.00606/) (2025)
- [Not All Tokens Are What You Need In Thinking](https://mlsanity.com/paper/2505.17827/) (2025)
- [DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training](https://mlsanity.com/paper/2504.17565/) (2025)
- [Incentivizing LLMs to Self-Verify Their Answers](https://mlsanity.com/paper/2506.01369/) (2025)
- [How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Study](https://mlsanity.com/paper/2504.00829/) (2025)
- [Scaling over Scaling: Exploring Test-Time Scaling Plateau in Large Reasoning Models](https://mlsanity.com/paper/2505.20522/) (2025)
- [Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Framework](https://mlsanity.com/paper/2507.06829/) (2025)
- [MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Driving](https://mlsanity.com/paper/2602.21952/) (2026)
- [Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoning](https://mlsanity.com/paper/2506.04611/) (2025)
- [Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?](https://mlsanity.com/paper/2510.12680/) (2025)
- [Test-time Scaling Techniques in Theoretical Physics -- A Comparison of Methods on the TPBench Dataset](https://mlsanity.com/paper/2506.20729/) (2025)
- [Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows](https://mlsanity.com/paper/2506.03332/) (2025)
- [Think Twice: Branch-and-Rethink Reasoning Reward Model](https://mlsanity.com/paper/2510.23596/) (2025)
- [From Conflict to Consensus: Boosting Medical Reasoning via Multi-Round Agentic RAG](https://mlsanity.com/paper/2603.03292/) (2026)
- [Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling](https://mlsanity.com/paper/2509.04474/) (2025)
- [BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents](https://mlsanity.com/paper/2510.23458/) (2025)
- [ThinkTwice: Jointly Optimizing Large Language Models for Reasoning and Self-Refinement](https://mlsanity.com/paper/2604.01591/) (2026)
- [Right Prediction, Wrong Reasoning: Uncovering LLM Misalignment in RA Disease Diagnosis](https://mlsanity.com/paper/2504.06581/) (2025)
- [SSR: Speculative Parallel Scaling Reasoning in Test-time](https://mlsanity.com/paper/2505.15340/) (2025)
- [MOTIF: Modular Thinking via Reinforcement Fine-tuning in LLMs](https://mlsanity.com/paper/2507.02851/) (2025)
- [ReThinker: Scientific Reasoning by Rethinking with Guided Reflection and Confidence Control](https://mlsanity.com/paper/2602.04496/) (2026)
- [Understanding Performance Gap Between Parallel and Sequential Sampling in Large Reasoning Models](https://mlsanity.com/paper/2604.05868/) (2026)
- [OptScale: Probabilistic Optimality for Inference-time Scaling](https://mlsanity.com/paper/2506.22376/) (2025)
- [Closed-Loop Transformers: Autoregressive Modeling as Iterative Latent Equilibrium](https://mlsanity.com/paper/2511.21882/) (2025)
- [Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning](https://mlsanity.com/paper/2505.14684/) (2025)
- [Double-Checker: Enhancing Reasoning of Slow-Thinking LLMs via Self-Critical Fine-Tuning](https://mlsanity.com/paper/2506.21285/) (2025)
- [Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework](https://mlsanity.com/paper/2509.05007/) (2025)

## Abstract (arXiv)

Recent advances in large language models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have demonstrated the effectiveness of test-time scaling, where extended reasoning processes substantially enhance model performance. Despite this, current models are constrained by limitations in handling long texts and reinforcement learning (RL) training efficiency. To address these issues, we propose a simple yet effective test-time scaling approach Multi-round Thinking. This method iteratively refines model reasoning by leveraging previous answers as prompts for subsequent rounds. Extensive experiments across multiple models, including QwQ-32B and DeepSeek-R1, consistently show performance improvements on various benchmarks such as AIME 2024, MATH-500, GPQA-diamond, and LiveCodeBench. For instance, the accuracy of QwQ-32B improved from 80.3% (Round 1) to 82.1% (Round 2) on the AIME 2024 dataset, while DeepSeek-R1 showed a similar increase from 79.7% to 82.0%. These results confirm that Multi-round Thinking is a broadly applicable, straightforward approach to achieving stable enhancements in model performance, underscoring its potential for future developments in test-time scaling techniques. The key prompt: {Original question prompt} The assistant's previous answer is: <answer> {last round answer} </answer>, and please re-answer.