Rate-Resolved Text Evaluation with Masked Diffusion Language Models
On the map: Diffusion models → Discrete and text diffusion
Abstract (arXiv)
A single evaluation score can rank generation systems but rarely reveals what failed. We propose rate-resolved text evaluation, which turns the masking rate of masked diffusion language models into a diagnostic axis. By reconstructing text under different fractions of masked tokens, one diffusion score becomes a vector of band-conditioned scores that reflect reconstruction difficulty at varying levels of visible context. On SummEval, preserving this rate structure yields complementary signal: the best masking-rate band improves fusion with autoregressive likelihood over a collapsed diffusion scalar, survives a sample-shuffle control that breaks rate ordering, and adds residual signal to a fixed prompted judge. A second diffusion model reproduces the effect more weakly. On WMT MQM zh-en, roundtrip mask-and-reconstruct probing produces a tunable localization curve in which ranking-oriented AUROC and span-detection F1 peak at different masking rates. Factuality benchmarks bound the interpretation: band semantics are model- and task-dependent, so each application requires empirical validation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 20 of its 34 references that are in the map. First posted on arXiv 11 Oct 2026.
Builds on
20 of 34 references.
Closest ancestors · 4
Same direction, newest first
- 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation16 May
- 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!8 May
- 2022Self-conditioned Embedding Diffusion for Text Generation8 Nov
- 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models17 Oct
Other references · 16
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
- 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
- 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
- 2021BARTScore: Evaluating Generated Text as Text Generationcited by 722
- 2020SummEval: Re-evaluating Summarization Evaluationcited by 674
- 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
- 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
4 more
- 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
- 2019MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distancecited by 418
- 2020Fill in the BLANC: Human-free quality estimation of document summariescited by 82
- 2022T5Score: Discriminative Fine-tuning of Generative Evaluation Metricscited by 19
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026NAVIRA: Decoupled Stochastic Remasking for Masked Diffusion Language Models
- 2026DiffScore: Text Evaluation Beyond Autoregressive Likelihood
- 2024Scaling up Masked Diffusion Models on Text
- 2026Representation-based Masked Diffusion Model
- 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling
- 2026Remask, Don't Replace: Token-to-Mask Refinement in Diffusion Language Models
- 2023A Cheaper and Better Diffusion Language Model with Soft-Masked Noise
- 2026Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching