mlsanityData to 14 Oct 2026

A Survey on Parallel Reasoning

arXiv 2510.12164 ↗First posted 14 Oct 2025cs.CLcited by 11

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

With the increasing capabilities of Large Language Models (LLMs), parallel reasoning has emerged as a new inference paradigm that enhances reasoning robustness by concurrently exploring multiple lines of thought before converging on a final answer. It has become a significant trend to explore parallel reasoning to overcome the fragility of standard sequential methods and improve practical performance. In this paper, we aim to survey and summarize the progress and challenges of parallel reasoning. We first present a formal definition of parallel reasoning and clarify its distinction from related concepts like Chain-of-Thought. Then, we organize and discuss advanced techniques based on a novel taxonomy, including non-interactive reasoning, interactive reasoning, and efficiency-focused decoding strategies. Additionally, we explore various application scenarios, such as solving complex problems and enhancing the reliability of LLM outputs.Finally, we highlight the core challenges of parallel reasoning and suggest potential directions for future research. We hope that our work can provide a useful roadmap for beginners and encourage more research on improving parallel reasoning methods. Related source can be avaliable in https://github.com/PPPP-kaqiu/Awesome-Parallel-Reasoning.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 61 of its 118 references that are in the map. First posted on arXiv 14 Oct 2025.

Builds on

61 of 118 references.

Closest ancestors · 18

Same direction, newest first

  1. 2025A2R: An Asymmetric Two-Stage Reasoning Framework for Parallel Reasoning26 Sep
  2. 2025Latency and Token-Aware Test-Time Compute11 Sep
  3. 2025Parallel-R1: Towards Parallel Thinking via Reinforcement Learning9 Sep
  4. 2025ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Compute30 Aug
  5. 2025Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs27 Aug
  6. 2025Deep Think with Confidence21 Aug
  7. 2025Confidence-Weighted Token Set Cover for Early Hypothesis Pruning in Self-Consistency6 Aug
  8. 2025Adaptive Termination for Multi-round Parallel Reasoning: An Universal Semantic Entropy-Guided Framework9 Jul
  9. 2025Learning to Reason Across Parallel Samples for LLM Reasoning10 Jun
  10. 2025SSR: Speculative Parallel Scaling Reasoning in Test-time21 May
  11. 2025Group Think: Multiple Concurrent Reasoning Agents Collaborating at Token Level Granularity16 May
  12. 2025Learning from Peers in Reasoning Models12 May
6 more
  1. 2025Accelerating Large Language Model Reasoning via Speculative Search3 May
  2. 2025Learning Adaptive Parallel Reasoning with Language Models21 Apr
  3. 2025Atom of Thoughts for Markov LLM Test-Time Scaling17 Feb
  4. 2025PairJudge RM: Perform Best-of-N Sampling with Knockout Tournament22 Jan
  5. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  6. 2022ThinkSum: Probabilistic reasoning over sets using large language models4 Oct

Other references · 43

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  7. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  8. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  9. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
  10. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  11. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  12. 2025Kimi K2: Open Agentic Intelligencecited by 567
31 more
  1. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  2. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 331
  3. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  4. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
  5. 2023An LLM Compiler for Parallel Function Callingcited by 179
  6. 2022Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generationcited by 171
  7. 2023Accelerating LLM Inference with Staged Speculative Decodingcited by 151
  8. 2023REST: Retrieval-Based Speculative Decodingcited by 150
  9. 2024Hydra: Sequentially-Dependent Draft Heads for Medusa Decodingcited by 117
  10. 2023Cascade Speculative Drafting for Even Faster LLM Inferencecited by 83
  11. 2024MALT: Improving Reasoning with Multi-Agent LLM Trainingcited by 76
  12. 2024SPIQA: A Dataset for Multimodal Question Answering on Scientific Paperscited by 75
  13. 2024SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Accelerationcited by 72
  14. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  15. 2024Kangaroo: Lossless Self-Speculative Decoding via Double Early Exitingcited by 58
  16. 2023PaSS: Parallel Speculative Samplingcited by 49
  17. 2025Hogwild! Inference: Parallel LLM Generation via Concurrent Attentioncited by 32
  18. 2024Multi-Candidate Speculative Decodingcited by 31
  19. 2025Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generationcited by 30
  20. 2025Learning to Keep a Promise: Scaling Language Model Decoding Parallelism with Learned Asynchronous Decodingcited by 28
  21. 2025ParallelSearch: Train your LLMs to Decompose Query and Search Sub-queries in Parallel with Reinforcement Learningcited by 26
  22. 2024APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decodingcited by 24
  23. 2024Asynchronous LLM Function Callingcited by 18
  24. 2024Enhancing LLM Reasoning with Multi-Path Collaborative Reactive and Reflection agentscited by 15
  25. 2025Enhancing Reasoning through Process Supervision with Monte Carlo Tree Searchcited by 14
  26. 2025ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMscited by 11
  27. 2025Speculate, then Collaborate: Fusing Knowledge of Language Models during Decodingcited by 8
  28. 2025MIRAGE: Scaling Test-Time Inference with Parallel Graph-Retrieval-Augmented Reasoning Chainscited by 7
  29. 2025Self-Reflective Reinforcement Learning for Diffusion-based Image Reasoning Generationcited by 6
  30. 2025Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decodingcited by 4
  31. 2025Hybrid Deep Searcher: Scalable Parallel and Sequential Search Reasoningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.

  1. 2025ParallelMuse: Agentic Parallel Thinking for Deep Information Seekingcited by 14
  2. 2025Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learningcited by 12
  3. 2026Policy of Thoughts: Scaling Test-Time Training for LLM Reasoning via Online Policy Evolutioncited by 3
  4. 2026CAPS: Cascaded Adaptive Pairwise Selection for Efficient Parallel Reasoningcited by 2
  5. 2026Share More, Search Less: Collaborative Parallel Thinking for Efficient Test-Time Scalingcited by 2
  6. 2026MedVerse: Efficient and Reliable Medical Reasoning via DAG-Structured Parallel Executioncited by 1
  7. 2026Asking the Right Questions: Improving Reasoning with Generated Stepping Stonescited by 1
  8. 2026OPE: Overcoming Information Saturation in Parallel Thinking via Outline-Guided Path Explorationcited by 0
  9. 2026Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoningcited by 0
  10. 2026Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoningcited by 0
  11. 2026Parason: Revealing Subtask and Trial Parallelism in LLM Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
  2. 2026Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning
  3. 2025Dynamic Parallel Tree Search for Efficient LLM Reasoning
  4. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  5. 2026Message Passing Enables Efficient Reasoning
  6. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
  7. 2025Efficient Reasoning Models: A Survey
  8. 2025Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Models