mlsanityData to 14 Oct 2026

DA-DLM: Explicitly Modeling Token Dependencies in Diffusion Language Models

arXiv 2609.15070 ↗First posted 14 Sep 2026cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion Language Models (DLMs) generate text by iteratively denoising a masked sequence, independently predicting multiple tokens at each step. This conditional independence discards inter-token dependencies and degrades coherence-an issue that parallels the multi-modality problem in Non-Autoregressive Translation (NAT). Drawing on the Directed Acyclic Transformer (DAT), which tackles this problem in NAT via a Directed Acyclic Graph (DAG), we propose DA-DLM, a model that adapts DAG-based dependency modeling to DLMs' iterative setting through a position-oriented DAG design. The position-oriented DAG binds node groups to fixed output positions so that tokens fixed in earlier steps anchor neighboring predictions via learned transitions, and evolves with denoising to focus on remaining uncertainty as anchors accumulate. On language modeling, open-ended generation, and summarization, DA-DLM consistently outperforms Block Diffusion, especially under fewer denoising steps, and matches autoregressive models while preserving the parallel generation advantage. Our code is publicly available at https://github.com/jipy0222/DA-DLM.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 15 of its 30 references that are in the map. First posted on arXiv 14 Sep 2026.

Builds on

15 of 30 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026LLaDA2.1: Speeding Up Text Diffusion via Token Editing9 Feb
  2. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data6 Jun

Other references · 13

  1. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  2. 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
  3. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 470
  4. 2017Non-Autoregressive Neural Machine Translationcited by 463
  5. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 420
  6. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
  7. 2022Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 44
  8. 2022On the Learning of Non-Autoregressive Transformerscited by 27
  9. 2026DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMscited by 14
  10. 2022Viterbi Decoding of Directed Acyclic Transformer for Non-Autoregressive Machine Translationcited by 13
  11. 2023Fuzzy Alignments in Directed Acyclic Graph for Non-Autoregressive Machine Translationcited by 11
  12. 2023Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generationcited by 8
1 more
  1. 2023Optimizing Non-Autoregressive Transformers with Contrastive Learningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Language Models: An Experimental Analysis
  2. 2025A Survey on Diffusion Language Models
  3. 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching
  4. 2026Hierarchical Continuous Diffusion Language Models
  5. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion
  6. 2026Temporally-Resolved Token Attribution Reveals the Generation Dynamics of Diffusion Language Models
  7. 2025Sequential Diffusion Language Models
  8. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models