# Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner

arXiv: https://arxiv.org/abs/2510.03206 · HTML: https://mlsanity.com/paper/2510.03206/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning.
- Builds on 69 of its 78 references that are in the map.
- First posted on arXiv 3 Oct 2025.

## Placed in

- LLM reasoning / [Diffusion reasoning and test-time scaling](https://mlsanity.com/lr/diffusion-reasoning-and-test-time-scaling/)
- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 30 of 69)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Simplified and Generalized Masked Diffusion for Discrete Data](https://mlsanity.com/paper/2406.04329/) (2024)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://mlsanity.com/paper/2210.08933/) (2022)

## Built on by (most cited first, 30 of 32)

- [ELF: Embedded Language Flows](https://mlsanity.com/paper/2605.10938/) (2026)
- [CANDI: Hybrid Discrete-Continuous Diffusion Models](https://mlsanity.com/paper/2510.22510/) (2025)
- [Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization](https://mlsanity.com/paper/2510.08233/) (2025)
- [Soft-Masked Diffusion Language Models](https://mlsanity.com/paper/2510.17206/) (2025)
- [CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think](https://mlsanity.com/paper/2603.02547/) (2026)
- [Solve the Loop: Attractor Models for Language and Reasoning](https://mlsanity.com/paper/2605.12466/) (2026)
- [HRM-Text: Efficient Pretraining Beyond Scaling](https://mlsanity.com/paper/2605.20613/) (2026)
- [LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement Learning](https://mlsanity.com/paper/2602.01705/) (2026)
- [Beyond Hard Masks: Progressive Token Evolution for Diffusion Language Models](https://mlsanity.com/paper/2601.07351/) (2026)
- [DLLM Agent: See Farther, Run Faster](https://mlsanity.com/paper/2602.07451/) (2026)
- [Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference](https://mlsanity.com/paper/2602.22868/) (2026)
- [DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling](https://mlsanity.com/paper/2605.23605/) (2026)
- [One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models](https://mlsanity.com/paper/2604.18839/) (2026)
- [Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction](https://mlsanity.com/paper/2512.05092/) (2025)
- [Recursive Scaling in Masked Diffusion Models](https://mlsanity.com/paper/2606.18022/) (2026)
- [Masked Language Flow Models](https://mlsanity.com/paper/2606.27617/) (2026)
- [Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion](https://mlsanity.com/paper/2607.10951/) (2026)
- [GeoBlock: Inferring Block Granularity from Dependency Geometry in Diffusion Language Models](https://mlsanity.com/paper/2603.26675/) (2026)
- [Continuous Language Diffusion as a Decoder-Interface Problem](https://mlsanity.com/paper/2606.08810/) (2026)
- [Multi-Mask Diffusion Language Models for Few-Step Generation](https://mlsanity.com/paper/2607.19686/) (2026)
- [From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models](https://mlsanity.com/paper/2607.26504/) (2026)
- [ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks](https://mlsanity.com/paper/2609.29102/) (2026)
- [Latent-Augmented Discrete Diffusion Models](https://mlsanity.com/paper/2510.18114/) (2025)
- [Understanding and Accelerating the Training of Masked Diffusion Language Models](https://mlsanity.com/paper/2605.13026/) (2026)
- [A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Models](https://mlsanity.com/paper/2609.06324/) (2026)
- [Representation-based Masked Diffusion Model](https://mlsanity.com/paper/2609.12382/) (2026)
- [E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models](https://mlsanity.com/paper/2609.37533/) (2026)
- [Scaling and Distilling Text Embeddings for Better Diffusibility](https://mlsanity.com/paper/2610.01016/) (2026)
- [Hierarchical Continuous Diffusion Language Models](https://mlsanity.com/paper/2610.02193/) (2026)
- [Plan Canvas: Fixed Reasoning Regions for Continuous Language Flows](https://mlsanity.com/paper/2610.05815/) (2026)

## Abstract (arXiv)

Diffusion language models, especially masked discrete diffusion models, have achieved great success recently. While there are some theoretical and primary empirical results showing the advantages of latent reasoning with looped transformers or continuous chain-of-thoughts, continuous diffusion models typically underperform their discrete counterparts. In this paper, we argue that diffusion language models do not necessarily need to be in the discrete space. In particular, we prove that continuous diffusion models have stronger expressivity than discrete diffusions and looped transformers. We attribute the contradiction between the theoretical expressiveness and empirical performance to their practical trainability: while continuous diffusion provides intermediate supervision that looped transformers lack, they introduce additional difficulty decoding tokens into the discrete token space from the continuous representation space. We therefore propose Coevolutionary Continuous Discrete Diffusion (CCDD), which defines a joint multimodal diffusion process on the union of a continuous representation space and a discrete token space, leveraging a single model to simultaneously denoise in the joint space. By combining two modalities, CCDD is expressive with rich semantics in the latent space, as well as good trainability and sample quality with the help of explicit discrete tokens. We also propose effective architectures and advanced training/sampling techniques for CCDD, which reveals strong empirical performance in extensive language modeling experiments on real-world tasks.