# LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling

arXiv: https://arxiv.org/abs/2604.11748 · HTML: https://mlsanity.com/paper/2604.11748/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Discrete and text diffusion, one of 14 directions in Diffusion models.
- Builds on 51 of its 54 references that are in the map.
- First posted on arXiv 13 Apr 2026.

## Placed in

- Diffusion models / [Discrete and text diffusion](https://mlsanity.com/diffusion/discrete-and-text-diffusion/)

## Builds on (most cited first, 30 of 51)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Large Language Diffusion Models](https://mlsanity.com/paper/2502.09992/) (2025)
- [Simple and Effective Masked Diffusion Language Models](https://mlsanity.com/paper/2406.07524/) (2024)
- [Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution](https://mlsanity.com/paper/2310.16834/) (2023)
- [Dream 7B: Diffusion Large Language Models](https://mlsanity.com/paper/2508.15487/) (2025)
- [DiGress: Discrete Denoising diffusion for graph generation](https://mlsanity.com/paper/2209.14734/) (2022)
- [Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model](https://mlsanity.com/paper/2408.11039/) (2024)
- [A Continuous Time Framework for Discrete Denoising Models](https://mlsanity.com/paper/2205.14987/) (2022)
- [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://mlsanity.com/paper/2210.08933/) (2022)
- [Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models](https://mlsanity.com/paper/2503.09573/) (2025)

## Built on by (most cited first, 30 of 36)

- [ELF: Embedded Language Flows](https://mlsanity.com/paper/2605.10938/) (2026)
- [Continuous Diffusion Scales Competitively with Discrete Diffusion for Language](https://mlsanity.com/paper/2605.18530/) (2026)
- [Self-conditioned Flow Map Language Models via Fixed-point Flows](https://mlsanity.com/paper/2607.00714/) (2026)
- [Posterior Refinement: Fast Language Generation via Any-Order Flow Maps](https://mlsanity.com/paper/2606.24773/) (2026)
- [Scaling Categorical Flow Maps](https://mlsanity.com/paper/2605.07820/) (2026)
- [Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasoners](https://mlsanity.com/paper/2606.29150/) (2026)
- [AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling](https://mlsanity.com/paper/2608.02602/) (2026)
- [DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling](https://mlsanity.com/paper/2605.23605/) (2026)
- [Let It Be Simple: One-Step Action Generation for Vision-Language-Action Models](https://mlsanity.com/paper/2606.05737/) (2026)
- [Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoning](https://mlsanity.com/paper/2609.15903/) (2026)
- [Masked Language Flow Models](https://mlsanity.com/paper/2606.27617/) (2026)
- [Low Perplexity is Repetition: A One-Dimensional Self-Conditioning Attractor in Continuous Diffusion LMs](https://mlsanity.com/paper/2607.00588/) (2026)
- [Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusion](https://mlsanity.com/paper/2607.10951/) (2026)
- [ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings](https://mlsanity.com/paper/2608.23551/) (2026)
- [Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or One](https://mlsanity.com/paper/2609.04531/) (2026)
- [DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs](https://mlsanity.com/paper/2606.01024/) (2026)
- [Continuous Language Diffusion as a Decoder-Interface Problem](https://mlsanity.com/paper/2606.08810/) (2026)
- [Expanding Flow Maps](https://mlsanity.com/paper/2607.21585/) (2026)
- [From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models](https://mlsanity.com/paper/2607.26504/) (2026)
- [Simplex Diffusion Models](https://mlsanity.com/paper/2609.35553/) (2026)
- [StyleShield: Exposing the Fragility of AIGC Detectors through Continuous Controllable Style Transfer](https://mlsanity.com/paper/2605.00924/) (2026)
- [Sampling from Flow Language Models via Marginal-Conditioned Bridges](https://mlsanity.com/paper/2605.13681/) (2026)
- [How to Guide Your Language Flow](https://mlsanity.com/paper/2609.19356/) (2026)
- [One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusion](https://mlsanity.com/paper/2609.33698/) (2026)
- [Fine-Tuning Diffusion Language Models with Context Selection and Target Weighting](https://mlsanity.com/paper/2609.38385/) (2026)
- [Distribution Matching Distillation for Continuous Diffusion Language Models](https://mlsanity.com/paper/2609.40235/) (2026)
- [Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Models](https://mlsanity.com/paper/2610.00894/) (2026)
- [Hierarchical Continuous Diffusion Language Models](https://mlsanity.com/paper/2610.02193/) (2026)
- [SOL: Measuring Gaps between Text Distributions by Double Sliced Wasserstein Metrics](https://mlsanity.com/paper/2610.06513/) (2026)
- [Representation-Space MMD for Diffusion Language Models](https://mlsanity.com/paper/2610.06648/) (2026)

## Abstract (arXiv)

Continuous diffusion has been the foundation of high-fidelity, controllable, and few-step generation of many data modalities such as images. However, in language modeling, prior continuous diffusion language models (DLMs) lag behind discrete counterparts due to the sparse data space and the underexplored design space. In this work, we close this gap with LangFlow, the first continuous DLM to rival discrete diffusion, by connecting embedding-space DLMs to Flow Matching via Bregman divergence, alongside three key innovations: (1) we derive a novel ODE-based NLL bound for principled evaluation of continuous flow-based language models; (2) we propose an information-uniform principle for setting the noise schedule, which motivates a learnable noise scheduler based on a Gumbel distribution; and (3) we revise prior training protocols by incorporating self-conditioning, as we find it improves both likelihood and sample quality of embedding-space DLMs with effects substantially different from discrete diffusion. Putting everything together, LangFlow rivals top discrete DLMs on both the perplexity (PPL) and the generative perplexity (Gen. PPL), reaching a PPL of 30.0 on LM1B and 24.6 on OpenWebText. It even exceeds autoregressive baselines in zero-shot transfer on 4 out of 7 benchmarks. LangFlow provides the first clear evidence that continuous diffusion is a promising paradigm for language modeling. Homepage: https://github.com/nealchen2003/LangFlow