mlsanityData to 14 Oct 2026

Latent-Augmented Discrete Diffusion Models

arXiv 2510.18114 ↗First posted 20 Oct 2025cs.LG cs.AI stat.MLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Discrete diffusion models have emerged as a powerful class of models and a promising route to fast language generation, but practical implementations typically rely on factored reverse transitions ignoring cross-token dependencies and degrading few-step performance. We propose Latent-Augmented Discrete Diffusion (LADD), which introduces a learnable auxiliary latent channel and performs diffusion over the joint (token, latent) space. The latent variables provide an intermediate representation expressing joint structure while preserving tractable parameterizations. We instantiate LADD with continuous latents (Co-LADD) and discrete latents (Di-LADD), and study two inference schedules: a joint diffusion that denoises data and latents together, and a sequential diffusion that first resolves latents and then samples tokens conditionally. We derive ELBO-style objectives and analyze design choices that balance latent expressivity with diffusion compatibility. In experiments, LADD models yield improvements on unconditional generation metrics as compared to state-of-the-art masked discrete diffusion baselines, and are effective at lower sampling budgets, where unmasking many tokens per step is desirable.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 43 of its 80 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

43 of 80 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Self-Speculative Masked Diffusions4 Oct
  2. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner3 Oct
  3. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modeling1 Oct
  4. 2025Training Optimal Large Diffusion Language Models28 Sep
  5. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  6. 2025Cosmos: Compressed and Smooth Latent Space for Text Diffusion Modeling26 Jun
  7. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  8. 2025Large Language Diffusion Models14 Feb
  9. 2024Think While You Generate: Discrete Diffusion with Planned Denoising8 Oct
  10. 2024Discrete Copula Diffusion2 Oct
  11. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  12. 2024Simple and Effective Masked Diffusion Language Models11 Jun
4 more
  1. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  2. 2024TEncDM: Understanding the Properties of the Diffusion Model in the Space of Language Model Encodings29 Feb
  3. 2022Continuous diffusion for categorical data28 Nov
  4. 2022DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models28 Nov

Other references · 27

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2022Classifier-Free Diffusion Guidancecited by 5,847
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  9. 2016Pointer Sentinel Mixture Modelscited by 3,096
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  12. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
15 more
  1. 2021Variational Diffusion Modelscited by 1,153
  2. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  3. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  4. 2021Score-based Generative Modeling in Latent Spacecited by 636
  5. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  6. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 574
  7. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
  8. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
  9. 2024Discrete Flow Matchingcited by 317
  10. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
  11. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  12. 2025Diffusion Transformers with Representation Autoencoderscited by 295
  13. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesiscited by 45
  14. 2024DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latentscited by 20
  15. 2025Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modelingcited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
  2. 2022Latent Diffusion for Language Generation
  3. 2026Reasoning with Latent Tokens in Diffusion Language Models
  4. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
  5. 2026How to Train Your Latent Diffusion Language Model Jointly With the Latent Space
  6. 2026Representation-Space MMD for Diffusion Language Models
  7. 2026Breaking the Factorization Barrier in Diffusion Language Models
  8. 2026Towards Latent Diffusion Suitable For Text