mlsanityData to 7 Oct 2026

LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling

arXiv 2604.11748 ↗First posted 13 Apr 2026cs.CL cs.LGcited by 28

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Continuous diffusion has been the foundation of high-fidelity, controllable, and few-step generation of many data modalities such as images. However, in language modeling, prior continuous diffusion language models (DLMs) lag behind discrete counterparts due to the sparse data space and the underexplored design space. In this work, we close this gap with LangFlow, the first continuous DLM to rival discrete diffusion, by connecting embedding-space DLMs to Flow Matching via Bregman divergence, alongside three key innovations: (1) we derive a novel ODE-based NLL bound for principled evaluation of continuous flow-based language models; (2) we propose an information-uniform principle for setting the noise schedule, which motivates a learnable noise scheduler based on a Gumbel distribution; and (3) we revise prior training protocols by incorporating self-conditioning, as we find it improves both likelihood and sample quality of embedding-space DLMs with effects substantially different from discrete diffusion. Putting everything together, LangFlow rivals top discrete DLMs on both the perplexity (PPL) and the generative perplexity (Gen. PPL), reaching a PPL of 30.0 on LM1B and 24.6 on OpenWebText. It even exceeds autoregressive baselines in zero-shot transfer on 4 out of 7 benchmarks. LangFlow provides the first clear evidence that continuous diffusion is a promising paradigm for language modeling. Homepage: https://github.com/nealchen2003/LangFlow

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 51 of its 54 references that are in the map. First posted on arXiv 13 Apr 2026.

Builds on

51 of 54 references.

Closest ancestors · 16

Same direction, newest first

  1. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think3 Mar
  2. 2026Flow Map Language Models: One-step Language Modeling via Continuous Denoising18 Feb
  3. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation7 Oct
  4. 2025Dream 7B: Diffusion Large Language Models21 Aug
  5. 2025The Diffusion Duality12 Jun
  6. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  7. 2025Continuous Diffusion Model for Language Modeling17 Feb
  8. 2025Large Language Diffusion Models14 Feb
  9. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  10. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  11. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  12. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
4 more
  1. 2023Likelihood-Based Diffusion Language Models30 May
  2. 2022Continuous diffusion for categorical data28 Nov
  3. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov
  4. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2022Flow Matching for Generative Modelingcited by 5,426
  10. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
23 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  4. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  5. 2022Video Diffusion Modelscited by 2,280
  6. 2023Consistency Modelscited by 1,778
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  8. 2021Variational Diffusion Modelscited by 1,145
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  10. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  11. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  13. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  14. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  15. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
  16. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 148
  17. 2024Variational Flow Matching for Graph Generationcited by 78
  18. 2024Categorical Flow Matching on Statistical Manifoldscited by 43
  19. 2025Esoteric Language Models: A Family of Any-Order Diffusion LLMscited by 33
  20. 2025Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spacescited by 19
  21. 2025α-Flow: A Unified Framework for Continuous-State Discrete Flow Matching Modelscited by 17
  22. 2025Exponential Family Variational Flow Matching for Tabular Data Generationcited by 15
  23. 2025ShortListing Model: A Streamlined SimplexDiffusion for Discrete Variable Generationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 28.

28 papers.

Sort:

By area:

  1. 2026ELF: Embedded Language Flowscited by 27
  2. 2026Continuous Diffusion Scales Competitively with Discrete Diffusion for Languagecited by 13
  3. 2026Self-conditioned Flow Map Language Models via Fixed-point Flowscited by 9
  4. 2026Posterior Refinement: Fast Language Generation via Any-Order Flow Mapscited by 8
  5. 2026Scaling Categorical Flow Mapscited by 6
  6. 2026Flow Reasoning Models: Turning Flows Into Efficient Recurrent Reasonerscited by 5
  7. 2026DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modelingcited by 3
  8. 2026Let It Be Simple: One-Step Action Generation for Vision-Language-Action Modelscited by 3
  9. 2026AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modelingcited by 3
  10. 2026Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoningcited by 3
  11. 2026Masked Language Flow Modelscited by 2
  12. 2026Low Perplexity is Repetition: A One-Dimensional Self-Conditioning Attractor in Continuous Diffusion LMscited by 2
16 more
  1. 2026ConvergeFlow: Language Flow with Provable Convergence to Token Embeddingscited by 2
  2. 2026Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or Onecited by 2
  3. 2026DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMscited by 1
  4. 2026Expanding Flow Mapscited by 1
  5. 2026Simplex Diffusion Modelscited by 1
  6. 2026StyleShield: Exposing the Fragility of AIGC Detectors through Continuous Controllable Style Transfercited by 0
  7. 2026Sampling from Flow Language Models via Marginal-Conditioned Bridgescited by 0
  8. 2026Sticky Jump Diffusions: A Unifying View of Masked, Continuous, and Hybrid Diffusioncited by 0
  9. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Modelscited by 0
  10. 2026How to Guide Your Language Flowcited by 0
  11. 2026One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusioncited by 0
  12. 2026Fine-Tuning Diffusion Language Models with Context Selection and Target Weightingcited by 0
  13. 2026Distribution Matching Distillation for Continuous Diffusion Language Modelscited by 0
  14. 2026Clock Diffusion: Efficient Semi-Autoregressive Continuous Diffusion Language Modelscited by 0
  15. 2026Hierarchical Continuous Diffusion Language Modelscited by 0
  16. 2026Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Roundscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Denoising Hierarchical Representations: Joint Continuous Diffusion for Language Modeling
  2. 2026Diffusion Language Models: An Experimental Analysis
  3. 2026Language Modeling with Hyperspherical Flows
  4. 2026FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
  5. 2026Representation-Space MMD for Diffusion Language Models
  6. 2026Consistent Diffusion Language Models
  7. 2022Latent Diffusion for Language Generation
  8. 2025A Survey on Diffusion Language Models