mlsanityData to 14 Oct 2026

CT-OPD: Counterfactual Trace On-Policy Distillation for Diffusion Vision-Language Models

arXiv 2609.32781 ↗First posted 26 Sep 2026cs.CV cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Diffusion vision-language models generate answers by gradually resolving masked tokens, making accurate conditional prediction in partially resolved states central to post-training. Masking completed answers yields coherent contexts and targets, but prescribed masks do not reflect the model's reveal decisions. Its trajectories capture these decisions, yet their provisional visible tokens can conflict with the target response. Outcome-based reinforcement learning follows these trajectories but provides only response-level feedback, which loses contrast when sampled rewards tie. To align coherent token-level supervision with the model's reveal decisions, we introduce Counterfactual Trace On-Policy Distillation (CT-OPD), which combines completed teacher responses with trajectory masks from the current student. CT-OPD retokenizes each teacher response in the student's vocabulary and extracts unresolved-position masks at successive stages of the student's reverse process. For each mask, it discards provisional rollout values and reconstructs the partial state from the teacher endpoint, so the supervised positions follow the current trajectory while the visible context and targets remain consistent with the same response. The student is trained on these reconstructed states with its native categorical loss, and trajectories are refreshed as the model evolves. Across dense and sparse diffusion architectures, CT-OPD consistently enhances multimodal understanding and reasoning capabilities, with gains of up to 9.80 points on the nine-benchmark average. On the unified understanding-and-generation architecture, it also improves both visual understanding and image generation, showing that the same principle transfers across architectures and modalities. Ablations further attribute these gains to coherent reconstruction and current-model trajectory masks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 32 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

31 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Self-Rewarding Vision-Language Model via Reasoning Decomposition27 Aug

Other references · 30

  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  2. 2025Qwen-Image Technical Reportcited by 1,044
  3. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  4. 2025Large Language Diffusion Modelscited by 912
  5. 2016Sequence-Level Knowledge Distillationcited by 801
  6. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 640
  7. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
  8. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 250
  9. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
  10. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Modelscited by 88
  11. 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 84
  12. 2024Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMscited by 63
18 more
  1. 2025LLaDA-MoE: A Sparse MoE Diffusion Language Modelcited by 57
  2. 2025Universal Cross-Tokenizer Distillation via Approximate Likelihood Matchingcited by 41
  3. 2026d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillationcited by 37
  4. 2025MDPO: Overcoming the Training-Inference Divide of Masked Diffusion Language Modelscited by 32
  5. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimizationcited by 20
  6. 2025Consolidating Reinforcement Learning for Multimodal Discrete Diffusion Modelscited by 12
  7. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillationcited by 11
  8. 2025DiffusionVL: Translating Any Autoregressive Models into Diffusion Vision Language Modelscited by 10
  9. 2026Fast-dVLM: Efficient Block-Diffusion VLM via Direct Conversion from Autoregressive VLMcited by 9
  10. 2025A Dual-Space Framework for General Knowledge Distillation of Large Language Modelscited by 8
  11. 2026Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalizationcited by 8
  12. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 7
  13. 2026Trace-Based On-Policy Distillation for Masked Diffusion Language Modelscited by 7
  14. 2026dOPSD: On-Policy Self-Distillation for Diffusion Language Modelscited by 6
  15. 2026Diffusion-State Policy Optimization for Masked Diffusion Language Modelscited by 4
  16. 2026Beyond Mode-Seeking RL: Trajectory-Balance Post-Training for Diffusion Language Modelscited by 3
  17. 2026OPTD: On-Policy Transition Distillation with Consistency-Guided Adaptive Compression for Few-Step Diffusion Language Modelscited by 3
  18. 2026BARD: Bridging AutoRegressive and Diffusion Vision-Language Models Via Highly Efficient Progressive Block Merging and Stage-Wise Distillationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026VC-OPD: Visual Counterfactual On-Policy Distillation for Grounded Vision-Language Reasoning
  2. 2026Dense Supervision, Sparse Updates: On the Sparsity and Geometry of On-Policy Distillation
  3. 2026DiffusionOPD: A Unified Perspective of On-Policy Distillation in Diffusion Models
  4. 2026Towards Unbiased On-Policy Distillation for Block Diffusion Language Models
  5. 2026STEP-OPD: Rethinking Output Targets and Internal Dynamics in On-Policy Distillation for Diffusion Models
  6. 2026TTRSD: Test-Time Reinforcement Learning with Self-Distillation for Vision-Language Models
  7. 2026Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance
  8. 2026Mask-Aware Policy Gradients for Diffusion Language Models