# CANDI: Hybrid Discrete-Continuous Diffusion Models

arXiv: https://arxiv.org/abs/2510.22510 · HTML: https://mlsanity.com/paper/2510.22510/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 41 of its 60 references that are in the map.
- First posted on arXiv 26 Oct 2025.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 30 of 41)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024)
- [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023)
- [Simplified and Generalized Masked Diffusion for Discrete Data](https://mlsanity.com/paper/2406.04329/) (2024)
- [Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions](https://mlsanity.com/paper/2102.05379/) (2021)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models](https://mlsanity.com/paper/2503.09573/) (2025)
- [Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data](https://mlsanity.com/paper/2406.03736/) (2024)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [Continuous diffusion for categorical data](https://mlsanity.com/paper/2211.15089/) (2022)
- [Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling](https://mlsanity.com/paper/2409.02908/) (2024)
- [Scaling up Masked Diffusion Models on Text](https://mlsanity.com/paper/2410.18514/) (2024)
- [SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control](https://mlsanity.com/paper/2210.17432/) (2022)
- [Score-based Continuous-time Discrete Diffusion Models](https://mlsanity.com/paper/2211.16750/) (2022)
- [Simple Guidance Mechanisms for Discrete Diffusion Models](https://mlsanity.com/paper/2412.10193/) (2024)
- [Likelihood-Based Diffusion Language Models](https://mlsanity.com/paper/2305.18619/) (2023)
- [The Diffusion Duality](https://mlsanity.com/paper/2506.10892/) (2025)
- [Self-conditioned Embedding Diffusion for Text Generation](https://mlsanity.com/paper/2211.04236/) (2022)
- [Latent Diffusion for Language Generation](https://mlsanity.com/paper/2212.09462/) (2022)
- [Oops I Took A Gradient: Scalable Sampling for Discrete Distributions](https://mlsanity.com/paper/2102.04509/) (2021)

## Built on by (most cited first, 30 of 34)

- [ELF: Embedded Language Flows](https://mlsanity.com/paper/2605.10938/) (2026)
- [Discrete Flow Maps](https://mlsanity.com/paper/2604.09784/) (2026)
- [Continuous Diffusion Scales Competitively with Discrete Diffusion for Language](https://mlsanity.com/paper/2605.18530/) (2026)
- [Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone](https://mlsanity.com/paper/2512.22615/) (2025)
- [Generative Frontiers: Why Evaluation Matters for Diffusion Language Models](https://mlsanity.com/paper/2604.02718/) (2026)
- [CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think](https://mlsanity.com/paper/2603.02547/) (2026)
- [Self-conditioned Flow Map Language Models via Fixed-point Flows](https://mlsanity.com/paper/2607.00714/) (2026)
- [CoBit: Language Modeling with Bitstream Diffusion](https://mlsanity.com/paper/2605.07013/) (2026)
- [Scaling Categorical Flow Maps](https://mlsanity.com/paper/2605.07820/) (2026)
- [Spherical Flows for Sampling Categorical Data](https://mlsanity.com/paper/2605.05629/) (2026)
- [Mofasa: A Step Change in Metal-Organic Framework Generation](https://mlsanity.com/paper/2512.01756/) (2025)
- [Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?](https://mlsanity.com/paper/2604.02028/) (2026)
- [On the Role of Discreteness in Diffusion LLMs](https://mlsanity.com/paper/2512.22630/) (2025)
- [Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference](https://mlsanity.com/paper/2602.22868/) (2026)
- [Discrete Stochastic Localization for Non-autoregressive Generation](https://mlsanity.com/paper/2602.16169/) (2026)
- [MetaState: Persistent Working Memory Enhances Reasoning in Discrete Diffusion Language Models](https://mlsanity.com/paper/2603.01331/) (2026)
- [Masked Language Flow Models](https://mlsanity.com/paper/2606.27617/) (2026)
- [Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion](https://mlsanity.com/paper/2607.10951/) (2026)
- [Simplex Relaxation for Discrete Diffusion](https://mlsanity.com/paper/2608.10615/) (2026)
- [Token Time Continuous Diffusion for Language Modeling](https://mlsanity.com/paper/2607.14106/) (2026)
- [Multi-Mask Diffusion Language Models for Few-Step Generation](https://mlsanity.com/paper/2607.19686/) (2026)
- [From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models](https://mlsanity.com/paper/2607.26504/) (2026)
- [Discrete Stochastic Localization for Non-autoregressive Generation](https://mlsanity.com/paper/2605.12836/) (2026)
- [Understanding and Accelerating the Training of Masked Diffusion Language Models](https://mlsanity.com/paper/2605.13026/) (2026)
- [BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers](https://mlsanity.com/paper/2606.02241/) (2026)
- [HybridCodec: Modeling Discrete and Continuous Representations for Efficient Speech Language Models](https://mlsanity.com/paper/2606.27627/) (2026)
- [Diffusion Language Model for Recommendation](https://mlsanity.com/paper/2607.21519/) (2026)
- [Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported](https://mlsanity.com/paper/2609.33947/) (2026)
- [Gumbel Straight Flow: Distilling Autoregressive Models into One-step Flow Maps](https://mlsanity.com/paper/2610.00497/) (2026)
- [Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models](https://mlsanity.com/paper/2610.00894/) (2026)

## Abstract (arXiv)

While continuous diffusion has shown remarkable success in continuous domains such as image generation, its direct application to discrete data has underperformed pure discrete formulations. To understand this gap, we introduce token identifiability, an analytical framework characterizing how Gaussian noise corrupts discrete data through two mechanisms: discrete identity corruption and continuous rank degradation. We reveal that these mechanisms scale differently with vocabulary size, creating a temporal dissonance that forces a tradeoff between learning continuous geometry and discrete structure. To address this, we propose CANDI (Continuous ANd DIscrete diffusion), a hybrid framework that decouples discrete and continuous corruption, enabling simultaneous learning of both. This unlocks the benefits of continuous diffusion for discrete spaces: on controlled generation, CANDI enables classifier-based guidance with off-the-shelf classifiers through simple gradient addition; on text generation, CANDI outperforms masked diffusion at low NFE, demonstrating the value of learning continuous gradients for discrete spaces. We include the code on the project page: https://patrickpynadath1.github.io/candi-lander.