mlsanityData to 14 Oct 2026

DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling

arXiv 2605.23605 ↗First posted 22 May 2026cs.LG cs.AI cs.CLcited by 4

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models intrinsically fail to capture correlations between decoded tokens, which leads to a harsh trade-off between sampling quality and throughput. To solve this issue, we propose DiLaDiff, a variant of masked diffusion language models with three components: (1) a continuous latent space with semantic capabilities, learned by an auto-encoder fine-tuned from an existing masked diffusion language model; (2) a latent diffusion model learning the prior over the encoder distribution; (3) a consistency model distilling the learned prior into a few-step latent generative model. We show that, even without distillation, our latent-guided diffusion model outperforms the masked diffusion baseline while significantly accelerating inference. Consistency distillation further lowers the computational overhead of continuous diffusion, such that the latent is generated in negligible time compared to discrete decoding.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 48 of its 52 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

48 of 52 references.

Closest ancestors · 20

Same direction, newest first

  1. 2026LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling13 Apr
  2. 2026Flow Map Language Models: One-step Language Modeling via Continuous Denoising18 Feb
  3. 2026Scaling Beyond Masked Diffusion Language Models16 Feb
  4. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning6 Oct
  5. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner3 Oct
  6. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling1 Oct
  7. 2025Dream 7B: Diffusion Large Language Models21 Aug
  8. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling26 Jun
  9. 2025The Diffusion Duality12 Jun
  10. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  11. 2025Continuous Diffusion Model for Language Modeling17 Feb
  12. 2025Large Language Diffusion Models14 Feb
8 more
  1. 2024Energy-Based Diffusion Language Models for Text Generation28 Oct
  2. 2024Discrete Copula Diffusion2 Oct
  3. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  4. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  5. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  6. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  7. 2024TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings29 Feb
  8. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 28

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2022Video Diffusion Modelscited by 2,305
  10. 2023Consistency Modelscited by 1,822
  11. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  12. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
16 more
  1. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  2. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  3. 2025Mean Flows for One-step Generative Modelingcited by 561
  4. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  5. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusioncited by 459
  6. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
  7. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  8. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 422
  9. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 373
  10. 2024Discrete Flow Matchingcited by 317
  11. 2023Text Embeddings Reveal (Almost) As Much As Textcited by 192
  12. 2024Consistency Models Made Easycited by 144
  13. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 112
  14. 2024Categorical Flow Matching on Statistical Manifoldscited by 47
  15. 2025Terminal Velocity Matchingcited by 20
  16. 2025Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modelingcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or Onecited by 2
  2. 2026Continuous Language Diffusion as a Decoder-Interface Problemcited by 1
  3. 2026Why Do Few-Step Text Latents Fail When Image Latents Work? Non-Commitment at Sharp Categorical Readoutscited by 0
  4. 2026Scaling and Distilling Text Embeddings for Better Diffusibilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Latent Diffusion for Language Generation
  2. 2025Latent-Augmented Discrete Diffusion Models
  3. 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching
  4. 2026AURORA-LM: Autoencoding Unified Representation for Continuous-Latent Diffusion Language Modeling
  5. 2025Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning
  6. 2026CRoCoDiL: Continuous and Robust Conditioned Diffusion for Language
  7. 2026Representation-Space MMD for Diffusion Language Models
  8. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion