Representation-Space MMD for Diffusion Language Models
On the map: Diffusion models → Discrete and text diffusion
Abstract (arXiv)
We introduce a post-training method for diffusion language models (DLMs) that minimizes Maximum Mean Discrepancy (MMD) between generated and reference distributions in the feature space of a frozen pretrained DLM. To estimate MMD, we retain contextual features at individual token positions, obtaining multiple observations per sequence from a single extractor pass. We optimize this objective using policy gradients for discrete models and direct differentiation through generated latents for continuous models. In both cases, computing the loss directly from these features enables efficient post-training without full sampling trajectories or jointly trained auxiliary models. Experiments show lower generative perplexity at comparable entropy on OpenWebText and better accuracy-computation trade-offs on GSM8K. On 16B DMax-LLaDA2.0 models with hybrid masked-uniform diffusion, we increase decoding parallelism with similar or higher accuracy on math and code benchmarks.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. First posted on arXiv 5 Oct 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025A Survey on Diffusion Language Models
- 2026Enhancing Diffusion Language Models with Autoregressive Post-Training Weights
- 2026Semantic DLM+: Improving Diffusion Language Models through Bias-variance Trade-off in Transition Kernel Design
- 2026DiLaDiff: Distilled Latent-Augmented Diffusion for Language Modeling
- 2026Understanding and Accelerating the Training of Masked Diffusion Language Models
- 2026Diffusion Language Models: An Experimental Analysis
- 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
- 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models