OPD-V: Visual On-Policy Self-Distillation with Modality Balance
On the map: LLM reasoning → Multimodal reasoning
also in 1 more
Abstract (arXiv)
On-Policy Self-Distillation (OPSD) has become a standard post-training approach for improving visual reasoning in multimodal large language models (MLLMs). Existing methods draw privileged information from diverse input sources to guide self-distillation. Yet these designs overlook Modality Imbalance, a challenge inherent to MLLM reasoning. When textual information dominates generation, the model cannot fully integrate its multimodal input. Consequently, carefully designed privileged information remains underused, limiting the effectiveness of OPSD. To examine this limitation, we construct a Positive Teacher with the Zoom-In Image and a Negative Teacher with the Mask Image, which exhibit different degrees of Modality Imbalance. Changes in their reasoning correctness and token logits reveal that Modality Balance can itself serve as privileged information. Motivated by this finding, we introduce OPD-V, a visual OPSD paradigm that instantiates such information through the Positive Teacher and Negative Teacher. Positive Modality-Balance Logits Margins define a Modality-Balance Trust Region that selects the on-policy tokens used for self-distillation. Experiments across 6 benchmarks, 4 MLLM backbones, and 5 post-training methods show that OPD-V consistently improves reasoning performance while reducing training cost.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 46 references that are in the map. First posted on arXiv 5 Aug 2026.
Builds on
39 of 46 references.
Closest ancestors · 7
Same direction, newest first
- 2026Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation17 Jun
- 2026Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning17 Jun
- 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning30 Dec
- 2025Thyme: Think Beyond Images15 Aug
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025Generalizing from SIMPLE to HARD Visual Reasoning: Can We Mitigate Modality Imbalance in VLMs?5 Jan
Other references · 32
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
- 2016Sequence-Level Knowledge Distillationcited by 761
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
- 2015Policy Distillationcited by 414
- 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
- 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 284
- 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
- 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 174
- 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 151
20 more
- 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
- 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoningcited by 82
- 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 71
- 2025PRISM: Self-Pruning Intrinsic Selection Method for Training-Free Multimodal Data Selectioncited by 57
- 2024LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steeringcited by 50
- 2025CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Processcited by 50
- 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 47
- 2025ASCD: Attention-Steerable Contrastive Decoding for Reducing Hallucination in MLLMcited by 36
- 2025Evaluating and Steering Modality Preferences in Multimodal Large Language Modelcited by 33
- 2026Visual-Advantage On-Policy Distillation for Vision-Language Modelscited by 22
- 2025Self-Evolving Multi-Agent Systems via Textual Backpropagationcited by 21
- 2023SPOT! Revisiting Video-Language Models for Event Understandingcited by 20
- 2025Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifierscited by 14
- 2025Can Visual Input Be Compressed? A Visual Token Compression Benchmark for Large Multimodal Modelscited by 11
- 2026Visual Contrastive Self-Distillationcited by 9
- 2025MINED: Probing and Updating with Multimodal Time-Sensitive Knowledge for Large Multimodal Modelscited by 7
- 2026Instruction Anchor: Dissecting the Mechanistic Dynamics of Modality Arbitrationcited by 6
- 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 5
- 2026EchoRL: Reinforcement Learning via Rollout Echoingcited by 3
- 2026ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoningcited by 2
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026Asymptotic Risk Calibration for Selective Question Answeringcited by 0
- 2026Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understandingcited by 0
- 2026PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillationcited by 0
- 2026On-Policy Visual Evidence Distillationcited by 0
- 2026EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perceptioncited by 0
- 2026Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillationcited by 0
- 2026Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenescited by 0
- 2026Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes1 Oct 2026
- 2026Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillation30 Sep 2026
- 2026EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception29 Sep 2026
- 2026On-Policy Visual Evidence Distillation29 Sep 2026
- 2026PE-OPSD: Internalizing Prompt Enhancement into Flow-matching Models via On-Policy Self-Distillation29 Sep 2026
- 2026Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding28 Sep 2026
- 2026Asymptotic Risk Calibration for Selective Question Answering12 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation
- 2026OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillation
- 2026ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation
- 2026H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation
- 2026Distilling Visual Reasoning into Text Space
- 2026Distill the Visual Evidence, Not Just the Answer: Cross-World On-Policy Distillation for Vision-Language Models
- 2026Fewer Tokens, More Self-Teaching: On-Policy Self-Distillation for Extreme Visual Token Reduction
- 2026V-Zero: Answer-Label-Free On-Policy Distillation with Contrastive Evidence Gating for Fine-Grained Visual Reasoning