mlsanityData to 7 Oct 2026

OPD-V: Visual On-Policy Self-Distillation with Modality Balance

arXiv 2608.05131 ↗First posted 5 Aug 2026cs.CV cs.AIcited by 7

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

On-Policy Self-Distillation (OPSD) has become a standard post-training approach for improving visual reasoning in multimodal large language models (MLLMs). Existing methods draw privileged information from diverse input sources to guide self-distillation. Yet these designs overlook Modality Imbalance, a challenge inherent to MLLM reasoning. When textual information dominates generation, the model cannot fully integrate its multimodal input. Consequently, carefully designed privileged information remains underused, limiting the effectiveness of OPSD. To examine this limitation, we construct a Positive Teacher with the Zoom-In Image and a Negative Teacher with the Mask Image, which exhibit different degrees of Modality Imbalance. Changes in their reasoning correctness and token logits reveal that Modality Balance can itself serve as privileged information. Motivated by this finding, we introduce OPD-V, a visual OPSD paradigm that instantiates such information through the Positive Teacher and Negative Teacher. Positive Modality-Balance Logits Margins define a Modality-Balance Trust Region that selects the on-policy tokens used for self-distillation. Experiments across 6 benchmarks, 4 MLLM backbones, and 5 post-training methods show that OPD-V consistently improves reasoning performance while reducing training cost.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 46 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

39 of 46 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation17 Jun
  2. 2026Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning17 Jun
  3. 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning30 Dec
  4. 2025Thyme: Think Beyond Images15 Aug
  5. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
  6. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  7. 2025Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs?5 Jan

Other references · 32

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  5. 2016Sequence-Level Knowledge Distillationcited by 761
  6. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
  7. 2015Policy Distillationcited by 414
  8. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
  9. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 284
  10. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
  11. 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 174
  12. 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
20 more
  1. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
  2. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoningcited by 82
  3. 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 71
  4. 2025PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selectioncited by 57
  5. 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 50
  6. 2025CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Processcited by 50
  7. 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 47
  8. 2025ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMcited by 36
  9. 2025Evaluating and Steering Modality Preferences in Multimodal Large Language Modelcited by 33
  10. 2026Visual-Advantage On-Policy Distillation for Vision-Language Modelscited by 22
  11. 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 21
  12. 2023SPOT! Revisiting Video-Language Models for Event Understandingcited by 20
  13. 2025Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifierscited by 14
  14. 2025Can Visual Input Be Compressed? A Visual Token Compression Benchmark for Large Multimodal Modelscited by 11
  15. 2026Visual Contrastive Self-Distillationcited by 9
  16. 2025MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Modelscited by 7
  17. 2026Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitrationcited by 6
  18. 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 5
  19. 2026EchoRL: Reinforcement Learning via Rollout Echoingcited by 3
  20. 2026ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation
  2. 2026OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation
  3. 2026ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation
  4. 2026H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation
  5. 2026Distilling Visual Reasoning into Text Space
  6. 2026Distill the Visual Evidence, Not Just the Answer: Cross-World On-Policy Distillation for Vision-Language Models
  7. 2026Fewer Tokens, More Self-Teaching: On-Policy Self-Distillation for Extreme Visual Token Reduction
  8. 2026V-Zero: Answer-Label-Free On-Policy Distillation with Contrastive Evidence Gating for Fine-Grained Visual Reasoning