mlsanityData to 14 Oct 2026

Parallel Continuous Chain-of-Thought with Jacobi Iteration

arXiv 2506.18582 ↗First posted 23 Jun 2025cs.CLcited by 25

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Continuous chain-of-thought has been shown to be effective in saving reasoning tokens for large language models. By reasoning with continuous latent thought tokens, continuous CoT is able to perform implicit reasoning in a compact manner. However, the sequential dependencies between latent thought tokens spoil parallel training, leading to long training time. In this paper, we propose Parallel Continuous Chain-of-Thought (PCCoT), which performs Jacobi iteration on the latent thought tokens, updating them iteratively in parallel instead of sequentially and thus improving both training and inference efficiency of continuous CoT. Experiments demonstrate that by choosing the proper number of iterations, we are able to achieve comparable or even better performance while saving nearly 50% of the training and inference time. Moreover, PCCoT shows better stability and robustness in the training process. Our code is available at https://github.com/whyNLP/PCCoT.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 21 references that are in the map. First posted on arXiv 23 Jun 2025.

Builds on

18 of 21 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation28 Feb
  2. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs17 Feb
  3. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step23 May
  4. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future27 Sep

Other references · 14

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  5. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  6. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
  7. 2024Break the Sequential Dependency of LLM Inference Using Lookahead Decodingcited by 298
  8. 2023Think before you speak: Training Language Models With Pause Tokenscited by 264
  9. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
  10. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representationscited by 201
  11. 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 162
  12. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 124
2 more
  1. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 61
  2. 2024Layer-Condensed KV Cache for Efficient Inference of Large Language Modelscited by 59

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 19.

25 papers.

Sort:

By area:

  1. 2025A Survey on Latent Reasoningcited by 81
  2. 2025LLM Latent Reasoning as Chain of Superpositioncited by 39
  3. 2025SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMscited by 22
  4. 2025Soft-Masked Diffusion Language Modelscited by 16
  5. 2025Parallel Test-Time Scaling for Latent Reasoning Modelscited by 14
  6. 2026Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structurecited by 11
  7. 2025KaVa: Latent Reasoning via Compressed KV-Cache Distillationcited by 8
  8. 2026LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoningcited by 8
  9. 2026SeLaR: Selective Latent Reasoning in Large Language Modelscited by 7
  10. 2026T^2MLR: Transformer with Temporal Middle-Layer Recurrencecited by 7
  11. 2026Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behaviorcited by 6
  12. 2026nabla-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Spacecited by 5
13 more
  1. 2026What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysiscited by 3
  2. 2026Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compressioncited by 2
  3. 2026Spectral-Progressive Thought Flow for Lightweight Multimodal Reasoningcited by 1
  4. 2026A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLMcited by 1
  5. 2026Assessing Adversarial Robustness of Latent Reasoning Modelscited by 1
  6. 2026AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Modelscited by 1
  7. 2026LoRi: Low-Rank Distillation for Implicit Reasoningcited by 0
  8. 2026Dynamic Semantic Compression for Efficient Latent-Space Inference in Large Language Modelscited by 0
  9. 2026Pretraining Latent Information Feedback Transformers with Teacher Supervisioncited by 0
  10. 2026Plan Canvas: Fixed Reasoning Regions for Continuous Language Flowscited by 0
  11. 2026RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Cancited by 0
  12. 2026Dynamic K-Head: Difficulty-Supervised Budget Selection for Continuous Latent Reasoningcited by 0
  13. 2026Rationale Distribution Distillation via Flow Matching for Distributional Continuous Latent Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Continuous Chain of Thought Enables Parallel Exploration and Reasoning
  2. 2026CIRF: Tokenizing Chain-of-Thoughts into Reusable Functional Units for Efficient Latent Reasoning in Large Language Models
  3. 2026Training Continuous Chain of Thought Models: A Tale of Two Regimes
  4. 2026Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought
  5. 2026Latent Reasoning with Supervised Thinking States
  6. 2025A Formal Comparison Between Chain of Thought and Latent Thought
  7. 2025Early Stopping Chain-of-thoughts in Large Language Models
  8. 2025Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought