mlsanityData to 14 Oct 2026

Rationale Distribution Distillation via Flow Matching for Distributional Continuous Latent Reasoning

arXiv 2610.19871 ↗First posted 14 Oct 2026cs.CL cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking also fits: Knowledge- and tool-augmented reasoning

Abstract (arXiv)

Sampling multiple chain-of-thought rationales from a language model can uncover correct solutions missed by a single attempt. However, these rationales are typically lengthy, making repeated autoregressive generation costly. Continuous latent reasoning offers a compact alternative, but point-valued methods condition repeated answer sampling on a single rationale, which can limit exploration of diverse paths and answer coverage. We thus propose rationale distribution distillation (RDD), training a conditional flow over fixed-length latent sequences by distilling from multiple textual reasoning paths. Furthermore, we expand the distillation targets through within-query mixup, interpolating between complete latent rationales for the same query. The flow nonautoregressively generates the entire latent sequence, and a frozen language model autoregressively decodes answers from sampled rationale contexts. Empirically, the resulting distributional reasoner achieves higher answer coverage than the evaluated point-valued latent reasoning baselines, particularly on advanced mathematical tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 47 of its 65 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

47 of 65 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers30 Jun
  2. 2026Are Latent Reasoning Models Easily Interpretable?6 Apr
  3. 2026GTS: Inference-Time Scaling of Latent Reasoning with a Learnable Gaussian Thought Sampler15 Feb
  4. 2025Parallel Test-Time Scaling for Latent Reasoning Models9 Oct
  5. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning6 Oct
  6. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains22 May
  7. 2025SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning16 May
  8. 2025Self-Training Elicits Concise Reasoning in Large Language Models27 Feb
  9. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuning13 Feb
  10. 2024Token-Budget-Aware LLM Reasoning24 Dec
  11. 2024Deliberation in Latent Space via Differentiable Cache Augmentation23 Dec
  12. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness16 Dec
1 more
  1. 2023Think before you speak: Training Language Models With Pause Tokens3 Oct

Other references · 34

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2022Flow Matching for Generative Modelingcited by 5,702
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  9. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  11. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  12. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
22 more
  1. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
  2. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  3. 2024LoRA+: Efficient Low Rank Adaptation of Large Modelscited by 377
  4. 2022Large Language Models Are Reasoning Teacherscited by 329
  5. 2024Discrete Flow Matchingcited by 317
  6. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  7. 2023Looped Transformers as Programmable Computerscited by 237
  8. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
  9. 2024Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durationscited by 170
  10. 2023Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Stepcited by 148
  11. 2025Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Spacecited by 117
  12. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 116
  13. 2020On Mixup Regularizationcited by 83
  14. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 78
  15. 2025Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thoughtcited by 77
  16. 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 70
  17. 2025LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinkingcited by 38
  18. 2025Continuous Chain of Thought Enables Parallel Exploration and Reasoningcited by 35
  19. 2023MCC-KD: Multi-CoT Consistent Knowledge Distillationcited by 28
  20. 2025Parallel Continuous Chain-of-Thought with Jacobi Iterationcited by 25
  21. 2026From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizonscited by 3
  22. 2025A Generalized Theory of Mixup for Structure-Preserving Synthetic Datacited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Latent Thought Flow: Efficient Latent Reasoning in Large Language Models
  2. 2026From Chain-of-Thought to Loops: Non-Autoregressive Latent Reasoning via Looped Transformers
  3. 2026What Matters for Latent Reasoning with Flow Matching
  4. 2026Next Thoughts Are Distributions: Generative Autoregressive Reasoning in the Latent Space
  5. 2026Closing the Loop on Latent Reasoning via Test-Time Reconstruction
  6. 2026CALR: Continuous Anchored Latent Reasoning via Render-of-Thought Compression
  7. 2026Latent Reasoning with Normalizing Flows
  8. 2026Recursive Offline Distillation via Entropy Gain Sampling and Answer Mixing