mlsanityData to 14 Oct 2026

Accelerate Parallelizable Reasoning via Parallel Decoding within One Sequence

arXiv 2503.20533 ↗First posted 26 Mar 2025cs.CLcited by 6

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Recent advances in reasoning models have demonstrated significant improvements in accuracy by employing detailed and comprehensive reasoning processes. However, generating these lengthy reasoning sequences is computationally expensive and time-consuming. To address this inefficiency, we leverage the inherent parallelizability of certain tasks to accelerate the reasoning process. Specifically, when multiple parallel reasoning steps exist, we decode multiple tokens per forward pass via a tree-like attention mask within a single sequence, avoiding additional memory usage. Experimental results show that our method achieves up to nearly 100% speedup in decoding while basically maintaining the answer quality.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 9 of its 17 references that are in the map. First posted on arXiv 26 Mar 2025.

Builds on

9 of 17 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation28 Jul

Other references · 8

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  3. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  5. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  6. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 297
  7. 2025FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Servingcited by 258
  8. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 124

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025Hogwild! Inference: Parallel LLM Generation via Concurrent Attentioncited by 32
  2. 2025ASPD: Unlocking Adaptive Serial-Parallel Decoding by Exploring Intrinsic Parallelism in LLMscited by 11
  3. 2025Asynchronous Reasoning: Training-Free Interactive Thinking LLMscited by 7
  4. 2025SSR: Speculative Parallel Scaling Reasoning in Test-timecited by 4
  5. 2026LACE: Lattice Attention for Cross-thread Explorationcited by 0
  6. 2026Line-Coupled Language Modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey on Parallel Reasoning
  2. 2025Less Is More: Fast and Accurate Reasoning with Cross-Head Unified Sparse Attention
  3. 2025DeepPrune: Parallel Scaling without Inter-trace Redundancy
  4. 2025ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models
  5. 2025Dynamic Parallel Tree Search for Efficient LLM Reasoning
  6. 2025Learning Adaptive Parallel Reasoning with Language Models
  7. 2025SPRINT: Enabling Interleaved Planning and Parallelized Execution in Reasoning Models
  8. 2025Hydra: A Modular Architecture for Efficient Long-Context Reasoning