mlsanityData to 13 Oct 2026

Who Should Lead Decoding Now? Tracking Reliable Trajectories for Ensembling Masked Diffusion Language Models

arXiv 2606.16281 ↗First posted 15 Jun 2026cs.CL cs.AIcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked Diffusion Language Models (MDLMs) have emerged as a distinct paradigm for sequence generation. As MDLMs become diverse in capabilities and knowledge coverage, an important question is how to combine their knowledge. Toward this, we first investigate the unique decoding dynamics of MDLMs. We find that successful generations exhibit stable confidence dynamics over answer-relevant positions, while unreliable trajectories can often be corrected by injecting promising intermediate states from other models. Guided by this observation, we propose TIE (Trajectory-based Iterative Ensembling), a knowledge fusion framework in which MDLMs iteratively identify reliable decoding trajectories and relay them across models. TIE tracks confidence dynamics over answer-relevant positions to determine which model currently follows a more reliable trajectory and selectively transfers partially denoised sequences across models. As the model on the more promising trajectory often changes across denoising steps, TIE allows different models to contribute complementary strengths at different stages of generation. Strong performance across diverse reasoning tasks, along with our analyses, suggests that TIE offers a practical approach to the underexplored problem of MDLM ensembling.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 18 of its 28 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

18 of 28 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025KLASS: KL-Guided Fast Inference in Masked Diffusion Models7 Nov
  2. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  3. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  4. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct

Other references · 14

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  2. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,798
  3. 2023Let's Verify Step by Stepcited by 3,672
  4. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,758
  5. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,685
  6. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 553
  7. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 426
  8. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 409
  9. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 302
  10. 2024Bridging the Gap between Different Vocabularies for LLM Ensemblecited by 45
  11. 2023Getting MoRE out of Mixture of Language Model Reasoning Expertscited by 44
  12. 2024Breaking the Ceiling of the LLM Community by Treating Token Generation as a Classification for Ensemblingcited by 33
2 more
  1. 2024Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensemblingcited by 28
  2. 2025Speculate, then Collaborate: Fusing Knowledge of Language Models during Decodingcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond Confidence: Adaptive and Coherent Decoding for Diffusion Language Models
  2. 2025Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models
  3. 2026On Trajectory-Aware Training for Masked Diffusion Language Models
  4. 2026Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space
  5. 2026CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
  6. 2026The Confidence Shortcut: A Reasoning Failure Mode of Masked Diffusion Models
  7. 2026Twist, Don't Tilt: Trajectory-Exact Constrained Decoding for Masked Diffusion Models
  8. 2025Beyond Static Cutoffs: One-Shot Dynamic Thresholding for Diffusion Language Models