What Gets Unmasked First? Trajectory Analysis of Diffusion Models for Graph-to-Text Generation
On the map: Diffusion models → Discrete and text diffusion
Abstract (arXiv)
We present the first systematic study of masked diffusion language models (MDLMs) for graph-to-text generation. We analyze MDLM generation trajectories -- the order in which tokens are unmasked during iterative decoding -- and find that, unlike autoregressive LLMs which generate text linearly, MDLMs naturally prioritize entities first, followed by relational and function words, with structural tokens resolved last. We further identify a previously undocumented failure mode of supervised fine-tuning: SFT disrupts this strategy by prematurely anchoring structural sentence-ending tokens early in the decoding trajectory, effectively fixing the output length which can lead to omitted or hallucinated information. To address this, we propose lambda-scaled structural decoding, a training-free inference-time modification that downweights structural token confidence and recovers +9.4 BLEU-4. Finally, we introduce Graph-LLaDA, which integrates a Graph Transformer encoder into LLaDA's decoding process to explicitly incorporate relational graph structure. Cross-dataset evaluation on LAGRANGE reveals that previous baselines overfit to dataset-specific patterns, while LLM- and MDLM-based approaches generalize significantly better.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 15 of its 37 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
15 of 37 references.
Closest ancestors · 3
Same direction, newest first
- 2025Taming Masked Diffusion Language Models via Consistency Trajectory Reinforcement Learning with Fewer Decoding Step28 Sep
- 2025Any-Order Flexible Length Masked Diffusion31 Aug
- 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct
Other references · 12
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
- 2019Text Generation from Knowledge Graphs with Graph Transformerscited by 122
- 2020Investigating Pretrained Language Models for Graph-to-Text Generationcited by 117
- 2021JointGT: Graph-Text Joint Representation Learning for Text Generation from Knowledge Graphscited by 58
- 2022GAP: A Graph-aware Language Model Framework for Knowledge Graph-to-Text Generationcited by 16
- 2023Evaluating Generative Models for Graph-to-Text Generationcited by 16
- 2020Promoting Graph Awareness in Linearized Graph-to-Text Generationcited by 14
- 2023Construction of Paired Knowledge Graph-Text Datasets Informed by Cyclic Evaluationcited by 6
- 2024MGSA: Multi-Granularity Graph Structure Attention for Knowledge Graph-to-Text Generationcited by 2
- 2026Is One Token All It Takes? Graph Pooling Tokens for LLM-based GraphQAcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models
- 2026Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models
- 2025Plan for Speed: Dilated Scheduling for Masked Diffusion Language Models
- 2026Constraints Are Graphs, Not Chains: Exact Decoding for Diffusion Language Models
- 2026TAG-DLM: Diffusion Language Models for Text-Attributed Graph Learning
- 2026x-Prediction Flow: Efficient Continuous Decoding for Masked Diffusion Language Models
- 2026CARVE: Verified Expansion for Variable-Length Generation in Diffusion Language Models
- 2024Scaling up Masked Diffusion Models on Text