mlsanityData to 14 Oct 2026

Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers

arXiv 2408.06195 ↗First posted 12 Aug 2024cs.CLcited by 138

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

This paper introduces rStar, a self-play mutual reasoning approach that significantly improves reasoning capabilities of small language models (SLMs) without fine-tuning or superior models. rStar decouples reasoning into a self-play mutual generation-discrimination process. First, a target SLM augments the Monte Carlo Tree Search (MCTS) with a rich set of human-like reasoning actions to construct higher quality reasoning trajectories. Next, another SLM, with capabilities similar to the target SLM, acts as a discriminator to verify each trajectory generated by the target SLM. The mutually agreed reasoning trajectories are considered mutual consistent, thus are more likely to be correct. Extensive experiments across five SLMs demonstrate rStar can effectively solve diverse reasoning problems, including GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA. Remarkably, rStar boosts GSM8K accuracy from 12.51% to 63.91% for LLaMA2-7B, from 36.46% to 81.88% for Mistral-7B, from 74.53% to 91.13% for LLaMA3-8B-Instruct. Code will be available at https://github.com/zhentingqi/rStar.

Key facts. Cited by 138 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 25 of its 43 references that are in the map. First posted on arXiv 12 Aug 2024.

Builds on

25 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022PAL: Program-aided Language Models18 Nov

Other references · 24

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2023Mistral 7Bcited by 3,164
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  6. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  7. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  10. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  11. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  12. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
12 more
  1. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  2. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  3. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solvingcited by 282
  4. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  5. 2023Take a Step Back: Evoking Reasoning via Abstraction in Large Language Modelscited by 159
  6. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 158
  7. 2024Self-Discover: Large Language Models Self-Compose Reasoning Structurescited by 110
  8. 2024Common 7B Language Models Already Possess Strong Math Capabilitiescited by 109
  9. 2023Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generationcited by 74
  10. 2023Self-Verification Improves Few-Shot Clinical Information Extractioncited by 52
  11. 2024MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Timecited by 41
  12. 2024Large Language Models Can Self-Correct with Key Condition Verificationcited by 39

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 19; 2025: 96; 2026: 23.

138 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
  3. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  4. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 263
  5. 2024Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Searchcited by 175
  6. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Designcited by 109
  7. 2025Reward-Guided Speculative Decoding for Efficient LLM Reasoningcited by 106
  8. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 91
  9. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 79
  10. 2024o1-Coder: an o1 Replication for Codingcited by 78
  11. 2025ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templatescited by 63
  12. 2025A Survey on (M)LLM-Based GUI Agentscited by 62
38 more
  1. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 60
  2. 2025SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoningcited by 59
  3. 2024Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTScited by 54
  4. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Searchcited by 53
  5. 2025When Thinking Fails: The Pitfalls of Reasoning for Instruction-Following in LLMscited by 53
  6. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 50
  7. 2025VLM-R3: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thoughtcited by 49
  8. 2025MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learningcited by 48
  9. 2025SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoningcited by 42
  10. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  11. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoningcited by 40
  12. 2025S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learningcited by 39
  13. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  14. 2025Search and Refine During Think: Facilitating Knowledge Refinement for Improved Retrieval-Augmented Reasoningcited by 35
  15. 2025Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physicscited by 34
  16. 2025ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewardingcited by 32
  17. 2025MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluationcited by 32
  18. 2025TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoningcited by 28
  19. 2025From Debate to Equilibrium: Belief-Driven Multi-Agent LLM Reasoning via Bayesian Nash Equilibriumcited by 26
  20. 2024RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Modelscited by 25
  21. 2025AStar: Boosting Multimodal Reasoning with Automated Structured Thinkingcited by 25
  22. 2025Tina: Tiny Reasoning Models via LoRAcited by 25
  23. 2025Table-R1: Inference-Time Scaling for Table Reasoningcited by 24
  24. 2025Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Modelscited by 23
  25. 2025From Passive to Active Reasoning: Can Large Language Models Ask the Right Questions under Incomplete Information?cited by 23
  26. 2025On the Emergence of Thinking in LLMs I: Searching for the Right Intuitioncited by 22
  27. 2025LEMMA: Learning from Errors for MatheMatical Advancement in LLMscited by 21
  28. 2025MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Searchcited by 21
  29. 2025DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversalcited by 20
  30. 2025IRIS: Interactive Research Ideation System for Accelerating Scientific Discoverycited by 19
  31. 2024Quantifying Generalization Complexity for Large Language Modelscited by 18
  32. 2024Towards a Unified View of Preference Learning for Large Language Models: A Surveycited by 17
  33. 2024DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Searchcited by 17
  34. 2025MMATH: A Multilingual Benchmark for Mathematical Reasoningcited by 17
  35. 2025Scaling Inference-Efficient Language Modelscited by 16
  36. 2025RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learningcited by 16
  37. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  38. 2025Enhancing Reasoning through Process Supervision with Monte Carlo Tree Searchcited by 14

50 most cited of 138 All 138 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Guiding Reasoning in Small Language Models with LLM Assistance
  2. 2025Enhancing Reasoning Capabilities of Small Language Models with Blueprints and Prompt Template Search
  3. 2024Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
  4. 2026Offline Guidance, Online Reasoning: Reusing LLM Feedback for Small Language Models
  5. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Models
  6. 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
  7. 2025Towards Reasoning Ability of Small Language Models
  8. 2026Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning