mlsanityData to 14 Oct 2026

CANDI: Hybrid Discrete-Continuous Diffusion Models

arXiv 2510.22510 ↗First posted 26 Oct 2025cs.LG stat.MLcited by 34

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

While continuous diffusion has shown remarkable success in continuous domains such as image generation, its direct application to discrete data has underperformed pure discrete formulations. To understand this gap, we introduce token identifiability, an analytical framework characterizing how Gaussian noise corrupts discrete data through two mechanisms: discrete identity corruption and continuous rank degradation. We reveal that these mechanisms scale differently with vocabulary size, creating a temporal dissonance that forces a tradeoff between learning continuous geometry and discrete structure. To address this, we propose CANDI (Continuous ANd DIscrete diffusion), a hybrid framework that decouples discrete and continuous corruption, enabling simultaneous learning of both. This unlocks the benefits of continuous diffusion for discrete spaces: on controlled generation, CANDI enables classifier-based guidance with off-the-shelf classifiers through simple gradient addition; on text generation, CANDI outperforms masked diffusion at low NFE, demonstrating the value of learning continuous gradients for discrete spaces. We include the code on the project page: https://patrickpynadath1.github.io/candi-lander.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 41 of its 60 references that are in the map. First posted on arXiv 26 Oct 2025.

Builds on

41 of 60 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner3 Oct
  2. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling1 Oct
  3. 2025Dream 7B: Diffusion Large Language Models21 Aug
  4. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling26 Jun
  5. 2025The Diffusion Duality12 Jun
  6. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  7. 2024Scaling up Masked Diffusion Models on Text24 Oct
  8. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  9. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  10. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  11. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  12. 2024TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings29 Feb
8 more
  1. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  2. 2023Likelihood-Based Diffusion Language Models30 May
  3. 2022SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers20 Dec
  4. 2022Latent Diffusion for Language Generation19 Dec
  5. 2022Empowering Diffusion Models on the Embedding Space for Text Generation19 Dec
  6. 2022Continuous diffusion for categorical data28 Nov
  7. 2022Self-conditioned Embedding Diffusion for Text Generation8 Nov
  8. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct

Other references · 21

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  8. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  10. 2021Variational Diffusion Modelscited by 1,153
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  12. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
9 more
  1. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 373
  2. 2022Score-based Continuous-time Discrete Diffusion Modelscited by 169
  3. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 155
  4. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  5. 2022A Langevin-like Sampler for Discrete Distributionscited by 44
  6. 2025Entropic Time Schedulers for Generative Diffusion Modelscited by 16
  7. 2024Gradient-based Discrete Sampling with Automatic Cyclical Schedulingcited by 9
  8. 2025Large Language Models to Diffusion Finetuningcited by 6
  9. 2024Improving Vector-Quantized Image Modeling with Latent Consistency-Matching Diffusioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 31.

34 papers.

Sort:

By area:

  1. 2026ELF: Embedded Language Flowscited by 45
  2. 2026Discrete Flow Mapscited by 29
  3. 2026Continuous Diffusion Scales Competitively with Discrete Diffusion for Languagecited by 19
  4. 2025Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbonecited by 18
  5. 2026Generative Frontiers: Why Evaluation Matters for Diffusion Language Modelscited by 16
  6. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Thinkcited by 14
  7. 2026Self-conditioned Flow Map Language Models via Fixed-point Flowscited by 14
  8. 2026CoBit: Language Modeling with Bitstream Diffusioncited by 10
  9. 2026Scaling Categorical Flow Mapscited by 8
  10. 2026Spherical Flows for Sampling Categorical Datacited by 6
  11. 2025Mofasa: A Step Change in Metal-Organic Framework Generationcited by 5
  12. 2026Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?cited by 5
22 more
  1. 2025On the Role of Discreteness in Diffusion LLMscited by 4
  2. 2026Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inferencecited by 4
  3. 2026Discrete Stochastic Localization for Non-autoregressive Generationcited by 3
  4. 2026MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Modelscited by 3
  5. 2026Masked Language Flow Modelscited by 2
  6. 2026Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusioncited by 2
  7. 2026Simplex Relaxation for Discrete Diffusioncited by 2
  8. 2026Token Time Continuous Diffusion for Language Modelingcited by 1
  9. 2026Multi-Mask Diffusion Language Models for Few-Step Generationcited by 1
  10. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 1
  11. 2026Discrete Stochastic Localization for Non-autoregressive Generationcited by 0
  12. 2026Understanding and Accelerating the Training of Masked Diffusion Language Modelscited by 0
  13. 2026BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplerscited by 0
  14. 2026HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Modelscited by 0
  15. 2026Diffusion Language Model for Recommendationcited by 0
  16. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reportedcited by 0
  17. 2026Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Mapscited by 0
  18. 2026Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Modelscited by 0
  19. 2026Hierarchical Continuous Diffusion Language Modelscited by 0
  20. 2026One-Step Text Generation by Seq-Driftingcited by 0
  21. 2026On the Design Space of Continuous Diffusion Language Modelscited by 0
  22. 2026Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking
  2. 2025Discrete State Diffusion Models: A Sample Complexity Perspective
  3. 2024Glauber Generative Model: Discrete Diffusion Models via Binary Classification
  4. 2024Discrete Modeling via Boundary Conditional Diffusion Processes
  5. 2025Why Masking Diffusion Works: Condition on the Jump Schedule for Improved Discrete Diffusion
  6. 2024Unified Discrete Diffusion for Categorical Data
  7. 2025Continuous Diffusion Model for Language Modeling
  8. 2026Uniform Discrete Diffusion Models are Minimax Optimal for Estimating Distributions with Small Effective Support Size