mlsanityData to 7 Oct 2026

A Ticket from Marginals to Joints: Coupled-Noise Distillation for One-Step Block Generation in Diffusion Language Models

arXiv 2609.06324 ↗First posted 6 Sep 2026cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Can a diffusion language model generate a coherent token block in one forward pass? Masked models already predict every position at once, but each prediction is the marginal distribution given the visible context, so the tokens can be mutually inconsistent and later steps revise those already committed. We introduce CONDOR (Coupled-Noise Distillation for One-Step Readout), trained from scratch to map different noise samples to different coherent blocks. Initially, random noise is not naturally paired with a target. Winner-take-all supervision lets different samples specialize, and self-distillation trains the one-pass output to match the refined coherent block. TinyStories experiments show diverse, coherent continuations over successive blocks, one forward pass each. Qualitative MNIST experiments show that the same approach can extend to multimodal generation, such as text-to-image and unconditional text-and-image generation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 64 of its 75 references that are in the map. First posted on arXiv 6 Sep 2026.

Builds on

64 of 75 references.

Closest ancestors · 27

Same direction, newest first

  1. 2026Don't Commit Alone: Joint Token Commitment in Diffusion Language Models5 Jul
  2. 2026Improved Large Language Diffusion Models24 Jun
  3. 2026Self-Generated Error Training for Token Editing in Diffusion Language Models15 Jun
  4. 2026DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs31 May
  5. 2026ELF: Embedded Language Flows11 May
  6. 2026Continuous Latent Diffusion Language Model7 May
  7. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Models12 Feb
  8. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editing9 Feb
  9. 2026CORE: Context-Robust Remasking for Diffusion Language Models4 Feb
  10. 2025Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner3 Oct
  11. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction1 Oct
  12. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models28 Sep
15 more
  1. 2025Dream 7B: Diffusion Large Language Models21 Aug
  2. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Models18 Aug
  3. 2025The Diffusion Duality12 Jun
  4. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  5. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  6. 2025Large Language Diffusion Models14 Feb
  7. 2024Energy-Based Diffusion Language Models for Text Generation28 Oct
  8. 2024Beyond Autoregression: Fast LLMs via Self-Distillation Through Time28 Oct
  9. 2024Discrete Copula Diffusion2 Oct
  10. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  11. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  12. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun
  13. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  14. 2022Latent Diffusion for Language Generation19 Dec
  15. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct

Other references · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  5. 2023Consistency Modelscited by 1,778
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  7. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  8. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  9. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  10. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  11. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  12. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 821
25 more
  1. 2016Sequence-Level Knowledge Distillationcited by 761
  2. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  3. 2015Generating Sentences from a Continuous Spacecited by 607
  4. 2025Mean Flows for One-step Generative Modelingcited by 524
  5. 2017Non-Autoregressive Neural Machine Translationcited by 457
  6. 2015Importance Weighted Autoencoderscited by 454
  7. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  8. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  9. 2017Toward Multimodal Image-to-Image Translationcited by 295
  10. 2023Accelerating Transformer Inference for Translation via Parallel Decodingcited by 123
  11. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 121
  12. 2018Fast Decoding in Sequence Models using Discrete Latent Variablescited by 103
  13. 2020Glancing Transformer for Non-Autoregressive Neural Machine Translationcited by 103
  14. 2020Fully Non-autoregressive Neural Machine Translation: Tricks of the Tradecited by 99
  15. 2016Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypothesescited by 96
  16. 2019Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Predictioncited by 96
  17. 2019FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flowcited by 89
  18. 2016Stochastic Multiple Choice Learning for Training Diverse Deep Ensemblescited by 77
  19. 2024CLLMs: Consistency Large Language Modelscited by 71
  20. 2019Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posteriorcited by 58
  21. 2024Distillation of Discrete Diffusion through Dimensional Correlationscited by 46
  22. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMscited by 42
  23. 2025Variational Autoencoding Discrete Diffusion with Enhanced Dimensional Correlations Modelingcited by 14
  24. 2025Parallel Token Prediction for Language Modelscited by 6
  25. 2025Multiple Choice Learning of Low-Rank Adapters for Language Modelingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
  2. 2026Multi-Mask Diffusion Language Models for Few-Step Generation
  3. 2023A Cheaper and Better Diffusion Language Model with Soft-Masked Noise
  4. 2026Infinite Mask Diffusion for Few-Step Distillation
  5. 2026Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
  6. 2026Auto-Regressive Masked Diffusion Models
  7. 2026DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
  8. 2026x-Prediction Flow: Efficient Continuous Decoding for Masked Diffusion Language Models