mlsanityData to 14 Oct 2026

VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation

arXiv 2510.23497 ↗First posted 27 Oct 2025cs.CVcited by 31

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

Training vision-language models (VLMs) for complex reasoning remains a challenging task, i.a. due to the scarcity of high-quality image-text reasoning data. Conversely, text-based reasoning resources are abundant and scalable, but it is still an open question how to leveraging them for VLM reasoning. To address this problem, we propose VOLD, a framework to transfer reasoning capabilities from text-only teacher models to VLM student models. To this end, VOLD combines reinforcement learning via Group Relative Policy Optimization (GRPO) with on-policy distillation, which allows the student reasoning traces to be guided by the teacher model, resulting in a significant gain over using GRPO alone. We further show that a cold-start alignment is essential for an effective transfer during the online training phase in this scenario and that without sufficient distributional alignment between teacher and student, on-policy distillation fails to provide meaningful guidance. We evaluate VOLD across diverse benchmarks including MMMU-Pro, MathVision, MathVista, and LogicVista, showing that VOLD outperforms the baseline model significantly and improves over the state of the art by a margin. Our ablation shows the importance of a cold-start alignment via SFT for on-policy distillation with a text-only teacher.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 21 of its 47 references that are in the map. First posted on arXiv 27 Oct 2025.

Builds on

21 of 47 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models10 Apr
  2. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  3. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
  4. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  5. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
  6. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 14

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  8. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  9. 2024OpenAI o1 System Cardcited by 1,244
  10. 2025Group Sequence Policy Optimizationcited by 835
  11. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
  12. 2025KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learningcited by 41
2 more
  1. 2025DCPO: Dynamic Clipping Policy Optimizationcited by 40
  2. 2025X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domainscited by 31

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 31.

31 papers.

Sort:

By area:

  1. 2026A Survey of On-Policy Distillation for Large Language Modelscited by 163
  2. 2026Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipecited by 49
  3. 2026Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillationcited by 32
  4. 2026SOD: Step-wise On-policy Distillation for Small Language Model Agentscited by 21
  5. 2026Prefix Teach, Suffix Fade: Local Teachability Collapse in Strong-to-Weak On-Policy Distillationcited by 16
  6. 2026Guava: Distilling Frontier VLMs into a Compact Agent through a Robotic Manipulation Harnesscited by 10
  7. 2026Visual Contrastive Self-Distillationcited by 10
  8. 2026Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RLcited by 8
  9. 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 7
  10. 2026A Formula-Driven Survey and Research Agenda for On-Policy Distillationcited by 6
  11. 2026Self-Supervised Visual On-Policy Distillationcited by 6
  12. 2026Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradientscited by 3
19 more
  1. 2026Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillationcited by 2
  2. 2026Deep Thought Alignment: Trajectory-Level Latent Distillation for Video Reasoningcited by 2
  3. 2026KEPO: Knowledge-Enhanced Preference Optimization for Multimodal Reasoning with Applications to Medical VQAcited by 1
  4. 2026Stage-1 Controls the Entropy Regime, Not the Outcomecited by 1
  5. 2026MIRROR: Learning from the Other View for Multi-Modal Reasoningcited by 1
  6. 2026OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolutioncited by 1
  7. 2026MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoningcited by 1
  8. 2026Unifying ICL, SFT, KL-Regularized RL Through a Bayesian Lenscited by 0
  9. 2026OPD-Aha: From Linguistic Momentum to Visual Reflection in Multimodal On-Policy Distillationcited by 0
  10. 2026From Self-Distillation to Self-Practice: Privileged Information for Multi-Turn Agentscited by 0
  11. 2026USA: Update-aware SAM for Cross-domain On-Policy Disitllation of Language Agentscited by 0
  12. 2026Look Before You Select: Rethinking Vocabulary Sparsification in On-Policy Distillationcited by 0
  13. 2026Counterfactual Attention Policy Distillation for Temporal Video Groundingcited by 0
  14. 2026EGSD: Event-Grounded Self-Distillation for Streaming Video Understandingcited by 0
  15. 2026Is Better Teacher Supervision Enough? Unlocking Student-side Learning in Multimodal On-Policy Distillationcited by 0
  16. 2026Same Reward, Different Skills: When Multimodal RL Learns to Lookcited by 0
  17. 2026Text-Centric Post-Training for Omni-Modal Reasoningcited by 0
  18. 2026SCAD: Structured Credit Assignment and Distillation for Long-Horizon Agentscited by 0
  19. 2026CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillation
  2. 2026VC-OPD: Visual Counterfactual On-Policy Distillation for Grounded Vision-Language Reasoning
  3. 2025ACPO: Adaptive Curriculum Policy Optimization for Aligning Vision-Language Models in Complex Reasoning
  4. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning
  5. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning
  6. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models
  7. 2026Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Grounding
  8. 2026Teaching the Way, Not the Answer: Privileged Tutoring Distillation for Multimodal Policy Optimization