# Multimodal distillation Area: [Knowledge distillation](https://mlsanity.com/distill/index.md). HTML: https://mlsanity.com/distill/multimodal-distillation/ ## Key facts - Multimodal distillation: 308 papers on arXiv since 2021 (data through 14 Oct 2026). - 25 of its 308 mapped papers (8%) cite “Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation” (2026, arXiv 2605.18740). - 12 of its 308 mapped papers (4%) cite “CLIP-KD: An Empirical Study of CLIP Model Distillation” (2023, arXiv 2307.12732). - 36 new papers 25 Sep–14 Oct. ## Most cited here - [Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation](https://mlsanity.com/paper/2605.18740/) (2026), cited by 25 papers here - [CLIP-KD: An Empirical Study of CLIP Model Distillation](https://mlsanity.com/paper/2307.12732/) (2023), cited by 12 papers here - [LLaVA-KD: A Framework of Distilling Multimodal Large Language Models](https://mlsanity.com/paper/2410.16236/) (2024), cited by 11 papers here - [Compressing Visual-linguistic Model via Knowledge Distillation](https://mlsanity.com/paper/2104.02096/) (2021), cited by 8 papers here - [The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation](https://mlsanity.com/paper/2206.06487/) (2022), cited by 8 papers here - [MoVE-KD: Knowledge Distillation for VLMs with Mixture of Visual Encoders](https://mlsanity.com/paper/2501.01709/) (2025), cited by 7 papers here - [Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillation](https://mlsanity.com/paper/2602.02994/) (2026), cited by 7 papers here - [Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection](https://mlsanity.com/paper/2108.03619/) (2021), cited by 6 papers here - [MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining](https://mlsanity.com/paper/2208.12262/) (2022), cited by 6 papers here - [VOLD: Reasoning Transfer from LLMs to Vision-Language Models via On-Policy Distillation](https://mlsanity.com/paper/2510.23497/) (2025), cited by 6 papers here ## New (25 Sep–14 Oct) - [LongForcing: Scaling Temporal Manifold Matching for Long-Horizon World Rollout](https://mlsanity.com/paper/2610.18289/) (2026-10-13) - [DVTR-KD: Distilling Vision-Language Models via Intermediate Visual Token Reconstruction](https://mlsanity.com/paper/2610.17853/) (2026-10-13) - [VC-OPD: Visual Counterfactual On-Policy Distillation for Grounded Vision-Language Reasoning](https://mlsanity.com/paper/2610.14939/) (2026-10-10) - [CV-OPD: Contrastive Vision On-policy Distillation](https://mlsanity.com/paper/2610.13264/) (2026-10-09) - [Rubric-CEPR: Self-Evolving Image Editing via Reward-Verified Self-Distillation](https://mlsanity.com/paper/2610.12469/) (2026-10-08) - [TAM: Task-Aware Memory Distillation for Efficient Spatiotemporal Prediction](https://mlsanity.com/paper/2610.11617/) (2026-10-08) - [CARE: Constrained Attention Refinement for Fine-Grained Visual Classification via Teacher-Student Distillation](https://mlsanity.com/paper/2610.11153/) (2026-10-08) - [Enhancing Long-Video VLM Embeddings with Query-Aware Streaming Latent Reasoning](https://mlsanity.com/paper/2610.04864/) (2026-10-04) - [KVE-KD: Key Visual Evidence-Guided Knowledge Distillation for Vision-Language Models](https://mlsanity.com/paper/2610.03842/) (2026-10-02) - [GRAFT: Growing Agglomerative Foundation Models via Continual Teacher Distillation](https://mlsanity.com/paper/2610.02597/) (2026-10-01) - [Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes](https://mlsanity.com/paper/2610.02117/) (2026-10-01) - [Video Evidence Indexing: Learning Where to Look from Video Previews for Token-Budgeted Long-Video Question Answering](https://mlsanity.com/paper/2610.00757/) (2026-09-30) - [Less Data, Better Timing: Student-Curriculum Coupling for VLM On-Policy Distillation in Temporal Video Grounding](https://mlsanity.com/paper/2609.40055/) (2026-09-30) - [Learning to Reason with Compressed Context: Ground-Truth-Free Adaptation of OmniLLMs via Self-Distillation](https://mlsanity.com/paper/2609.39953/) (2026-09-30) - [MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models](https://mlsanity.com/paper/2609.39920/) (2026-09-30) - [OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning](https://mlsanity.com/paper/2609.39490/) (2026-09-30) - [Distill the Visual Evidence, Not Just the Answer: Cross-World On-Policy Distillation for Vision-Language Models](https://mlsanity.com/paper/2609.38777/) (2026-09-30) - [UniEvo-VL: An On-policy Self-Distillation Training Recipe for Multimodal Model Self-improvement](https://mlsanity.com/paper/2609.38721/) (2026-09-30) - [LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation](https://mlsanity.com/paper/2610.00333/) (2026-09-29) - [LIFT: Layout-In-Future Video Generation under Large Viewpoint Change via On-Policy Self-Distillation](https://mlsanity.com/paper/2609.38146/) (2026-09-29) - [VISTA: Internalizing Collective Visual Experience via On-Policy Distillation for Active Multimodal Agents](https://mlsanity.com/paper/2609.38086/) (2026-09-29) - [RS-OPSD: Reliable Privileged On-Policy-Self-Distillation for Ultra-High-Resolution Remote Sensing VQA](https://mlsanity.com/paper/2609.38072/) (2026-09-29) - [Codebook-Guided Cross-Modal Knowledge Distillation for Structurally Heterogeneous Features](https://mlsanity.com/paper/2609.37243/) (2026-09-29) - [Spatial-OPSD: Self-Improving Spatial Reasoning via Label-Free Self-Distillation](https://mlsanity.com/paper/2609.37055/) (2026-09-29) - [EGSD: Event-Grounded Self-Distillation for Streaming Video Understanding](https://mlsanity.com/paper/2609.36803/) (2026-09-29) - [What Makes High-Magnification Knowledge Transferable? A Study of Cross-Resolution Distillation in Whole-Slide Imaging](https://mlsanity.com/paper/2609.36407/) (2026-09-29) - [Compress to Remember: Learning Compact Memory via On-Policy Distillation for Long Video Generation](https://mlsanity.com/paper/2609.36364/) (2026-09-28) - [Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding](https://mlsanity.com/paper/2609.34672/) (2026-09-28) - [Counterfactual Attention Policy Distillation for Temporal Video Grounding](https://mlsanity.com/paper/2609.34581/) (2026-09-28) - [Distilling Visual Reasoning into Text Space](https://mlsanity.com/paper/2609.34408/) (2026-09-28)