# CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think

arXiv: https://arxiv.org/abs/2603.02547 · HTML: https://mlsanity.com/paper/2603.02547/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning.
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 3 Mar 2026.

## Placed in

- LLM reasoning / [Diffusion reasoning and test-time scaling](https://mlsanity.com/lr/diffusion-reasoning-and-test-time-scaling/)
- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 17 of 17)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling](https://mlsanity.com/paper/1312.3005/) (2013)
- [Diffusion Transformers with Representation Autoencoders](https://mlsanity.com/paper/2510.11690/) (2025)
- [DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models](https://mlsanity.com/paper/2211.15029/) (2022)
- [Self-conditioned Embedding Diffusion for Text Generation](https://mlsanity.com/paper/2211.04236/) (2022)
- [A Survey on Diffusion Language Models](https://mlsanity.com/paper/2508.10875/) (2025)
- [TiDAR: Think in Diffusion, Talk in Autoregression](https://mlsanity.com/paper/2511.08923/) (2025)
- [Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling](https://mlsanity.com/paper/2510.01329/) (2025)
- [CANDI: Hybrid Discrete-Continuous Diffusion Models](https://mlsanity.com/paper/2510.22510/) (2025)
- [Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner](https://mlsanity.com/paper/2510.03206/) (2025)
- [TESS 2: A Large-Scale Generalist Diffusion Language Model](https://mlsanity.com/paper/2502.13917/) (2025)
- [LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning](https://mlsanity.com/paper/2510.04573/) (2025)
- [Sequential Diffusion Language Models](https://mlsanity.com/paper/2509.24007/) (2025)
- [Toward Diffusible High-Dimensional Latent Spaces: A Frequency Perspective](https://mlsanity.com/paper/2511.22249/) (2025)
- [Smoothie: Smoothing Diffusion on Token Embeddings for Text Generation](https://mlsanity.com/paper/2505.18853/) (2025)
- [Planner and Executor: Collaboration between Discrete Diffusion And Autoregressive Models in Reasoning](https://mlsanity.com/paper/2510.15244/) (2025)

## Built on by (most cited first, 14 of 14)

- [ELF: Embedded Language Flows](https://mlsanity.com/paper/2605.10938/) (2026)
- [LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling](https://mlsanity.com/paper/2604.11748/) (2026)
- [Language Modeling with Hyperspherical Flows](https://mlsanity.com/paper/2605.11125/) (2026)
- [How to Train Your Latent Diffusion Language Model Jointly With the Latent Space](https://mlsanity.com/paper/2605.07933/) (2026)
- [AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling](https://mlsanity.com/paper/2608.02602/) (2026)
- [Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacement](https://mlsanity.com/paper/2605.14368/) (2026)
- [Low Perplexity is Repetition: A One-Dimensional Self-Conditioning Attractor in Continuous Diffusion LMs](https://mlsanity.com/paper/2607.00588/) (2026)
- [Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One](https://mlsanity.com/paper/2609.04531/) (2026)
- [Continuous Language Diffusion as a Decoder-Interface Problem](https://mlsanity.com/paper/2606.08810/) (2026)
- [From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models](https://mlsanity.com/paper/2607.26504/) (2026)
- [Simplex Diffusion Models](https://mlsanity.com/paper/2609.35553/) (2026)
- [Why Do Few-Step Text Latents Fail When Image Latents Work? Non-Commitment at Sharp Categorical Readouts](https://mlsanity.com/paper/2606.30705/) (2026)
- [One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusion](https://mlsanity.com/paper/2609.33698/) (2026)
- [Hierarchical Continuous Diffusion Language Models](https://mlsanity.com/paper/2610.02193/) (2026)

## Abstract (arXiv)

We study why continuous diffusion language models (DLMs) have lagged behind discrete diffusion approaches despite their appealing continuous generative dynamics. Under a controlled token--recovery study, we identify token rounding, the final projection from denoised embeddings to tokens, as a primary bottleneck. Building on these insights, we propose CoDAR (Continuous Diffusion with Contextual AutoRegressive Decoder), a two--stage framework that keeps diffusion entirely continuous in an embedding space while learning a strong, context--conditional discretizer: an autoregressive Transformer decoder that cross--attends to the denoised embedding sequence and performs contextualized rounding to tokens. Experiments on LM1B and OpenWebText demonstrate that CoDAR substantially improves generation quality over latent diffusion and becomes competitive with strong discrete DLMs, while exposing a simple decoder--temperature knob to navigate the fluency--diversity trade off.