mlsanityData to 7 Oct 2026

Unified Energy for Invariant and Independent Decoding in Diffusion Language Models

arXiv 2606.09159 ↗First posted 8 Jun 2026cs.CL cs.AIcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Language Models (DLMs) enable parallel text generation by iteratively denoising a full sequence, offering attractive flexibility compared to auto-regressive (AR) decoding. However, existing methods fail to fully capture token relationships, leading to a performance gap relative to AR baselines, especially as the degree of parallelism increases. In this paper, we give a systematic analysis of the gap, identifying three key factors: (i) model capacity, (ii) dependency, and (iii) invariance. To address these issues, we first propose an invariant energy (Inv-E) together with an effective sampling-based estimator to handle the invariance issue. By further combining with the independent energy (Ind-E), we obtain a unified energy (Uni-E), that accounts for all these factors. Uni-E enjoys a unique advantage: it can be computed exactly without sampling-based partition estimation. Besides, Uni-E is model agnostic and can therefore be scaled to models of arbitrary size. We further prove that Uni-E can correct the distribution shift caused by dependency and invariance. Extensive experiments across Diffusion Language Models (DLMs) and Diffusion Large Language Models (DLLMs) demonstrate the effectiveness of the proposed Uni-E.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 42 of its 52 references that are in the map. First posted on arXiv 8 Jun 2026.

Builds on

42 of 52 references.

Closest ancestors · 18

Same direction, newest first

  1. 2026Breaking the Factorization Barrier in Diffusion Language Models9 Feb
  2. 2026CORE: Context-Robust Remasking for Diffusion Language Models4 Feb
  3. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100B10 Dec
  4. 2025Optimal Inference Schedules for Masked Diffusion Models6 Nov
  5. 2025Error Bounds and Optimal Schedules for Masked Diffusions with Factorized Approximations29 Oct
  6. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models8 Sep
  7. 2025Dream 7B: Diffusion Large Language Models21 Aug
  8. 2025A Survey on Diffusion Language Models14 Aug
  9. 2025The Diffusion Duality12 Jun
  10. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  11. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  12. 2025Large Language Diffusion Models14 Feb
6 more
  1. 2024Energy-Based Diffusion Language Models for Text Generation28 Oct
  2. 2024Discrete Copula Diffusion2 Oct
  3. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  4. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
  5. 2023A Survey of Diffusion Models in Natural Language Processing24 May
  6. 2021Autoregressive Diffusion Models5 Oct

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2016Pointer Sentinel Mixture Modelscited by 2,992
  10. 2023A Survey of Large Language Modelscited by 2,537
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  12. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
12 more
  1. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  2. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  3. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  4. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  5. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  6. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  7. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  8. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 302
  9. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decodingcited by 108
  10. 2023Model Agnostic Sample Reweighting for Out-of-Distribution Learningcited by 43
  11. 2025Self Speculative Decoding for Diffusion Large Language Modelscited by 29
  12. 2026DAWN: Dependency-Aware Fast Inference for Diffusion LLMscited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Locally Coherent Parallel Decoding in Diffusion Language Models
  2. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models
  3. 2026Rethinking the Generation Order of Block Diffusion Language Models
  4. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models
  5. 2025How Efficient Are Diffusion Language Models? A Critical Examination of Efficiency Evaluation Practices
  6. 2026Accelerating Diffusion Language Models via Structured Suffix Modeling
  7. 2026Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
  8. 2026LESS Is More: Mutual-Stability Sampling for Diffusion Language Models