mlsanityData to 7 Oct 2026

CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think

arXiv 2603.02547 ↗First posted 3 Mar 2026cs.CL cs.AI cs.LGcited by 13

On the map: LLM reasoning → Diffusion reasoning and test-time scaling

also in 1 more
Abstract (arXiv)

We study why continuous diffusion language models (DLMs) have lagged behind discrete diffusion approaches despite their appealing continuous generative dynamics. Under a controlled token--recovery study, we identify token rounding, the final projection from denoised embeddings to tokens, as a primary bottleneck. Building on these insights, we propose CoDAR (Continuous Diffusion with Contextual AutoRegressive Decoder), a two--stage framework that keeps diffusion entirely continuous in an embedding space while learning a strong, context--conditional discretizer: an autoregressive Transformer decoder that cross--attends to the denoised embedding sequence and performs contextualized rounding to tokens. Experiments on LM1B and OpenWebText demonstrate that CoDAR substantially improves generation quality over latent diffusion and becomes competitive with strong discrete DLMs, while exposing a simple decoder--temperature knob to navigate the fluency--diversity trade off.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Diffusion reasoning and test-time scaling, one of 17 directions in LLM reasoning. Builds on 17 of its 36 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

17 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025TiDAR: Think in Diffusion, Talk in Autoregression12 Nov
  2. 2025Planner and Executor: Collaboration between Discrete Diffusion And Autoregressive Models in Reasoning17 Oct
  3. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner3 Oct

Other references · 14

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  3. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  4. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  5. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Modelscited by 214
  6. 2022Self-conditioned Embedding Diffusion for Text Generationcited by 114
  7. 2025A Survey on Diffusion Language Modelscited by 75
  8. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 32
  9. 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 31
  10. 2025TESS 2: A Large-Scale Generalist Diffusion Language Modelcited by 30
  11. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  12. 2025Sequential Diffusion Language Modelscited by 19
2 more
  1. 2025Smoothie: Smoothing Diffusion on Token Embeddings for Text Generationcited by 8
  2. 2025Toward Diffusible High-Dimensional Latent Spaces: A Frequency Perspectivecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026LangFlow: Continuous Diffusion Rivals Discrete in Language Modelingcited by 28
  2. 2026ELF: Embedded Language Flowscited by 27
  3. 2026Language Modeling with Hyperspherical Flowscited by 15
  4. 2026How to Train Your Latent Diffusion Language Model Jointly With the Latent Spacecited by 8
  5. 2026AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modelingcited by 3
  6. 2026Low Perplexity is Repetition: A One-Dimensional Self-Conditioning Attractor in Continuous Diffusion LMscited by 2
  7. 2026Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or Onecited by 2
  8. 2026Where Should Diffusion Enter a Language Model? Geometry-Guided Hidden-State Replacementcited by 1
  9. 2026Simplex Diffusion Modelscited by 1
  10. 2026Why Do Few-Step Text Latents Fail When Image Latents Work? Non-Commitment at Sharp Categorical Readoutscited by 0
  11. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 0
  12. 2026One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusioncited by 0
1 more
  1. 2026Hierarchical Continuous Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Latent Diffusion for Language Generation
  2. 2025On the Role of Discreteness in Diffusion LLMs
  3. 2026ALoDLM: Adaptively Looped Diffusion Language Models
  4. 2026Continuous Diffusion Scales Competitively with Discrete Diffusion for Language
  5. 2026Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
  6. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  7. 2025Dream 7B: Diffusion Large Language Models
  8. 2026One Token Is Enough: Improving Diffusion Language Models with a Sink Token