mlsanityData to 14 Oct 2026

The Professor: Multi-Teacher Unsupervised Prompt Distillation for Vision-Language Models

arXiv 2606.23897 ↗First posted 22 Jun 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Prompt distillation compresses large vision-language models (VLMs) such as CLIP into lightweight student models by matching teacher predictions on unlabeled domain images. PromptKD (CVPR 2024) established this paradigm with a single PromptSRC-finetuned ViT-L/14 teacher and a ViT-B/16 student. We propose TheProfessor, a multi-teacher extension that distills from a fixed two-teacher ensemble: a domain-finetuned PromptSRC ViT-L/14 teacher and a zero-shot EVA-CLIP-L/14 teacher whose logits are pre-computed per dataset. We evaluate single-teacher PromptKD, equal-probability ensembling, and confidence-weighted ensembling on four base-to-novel datasets: Caltech-101, DTD, UCF101, and EuroSAT. In a 12-run single-seed sweep, confidence-weighted ensembling improves average HM from 87.52 to 89.28 (+1.77 points), while equal averaging improves average HM to 88.88 (+1.37 points). Gains are dataset dependent: they are negligible on Caltech-101 (+0.16 HM for confidence weighting), modest on UCF101 (+0.62), and largest on domain-shifted EuroSAT (+5.78). These results update our earlier Caltech-only analysis and show that multi-teacher prompt distillation is most useful when the second teacher contributes complementary supervision under domain shift.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 4 of its 24 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

4 of 24 references.

References · 4

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  3. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  4. 2023CLIP-KD: An Empirical Study of CLIP Model Distillationcited by 87

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024PromptKD: Unsupervised Prompt Distillation for Vision-Language Models
  2. 2025When Better Teachers Don't Make Better Students: Revisiting Knowledge Distillation for CLIP Models in VQA
  3. 2023Distilling Large Vision-Language Model with Out-of-Distribution Generalizability
  4. 2025MMT-ARD: Multimodal Multi-Teacher Adversarial Distillation for Robust Vision-Language Models
  5. 2023Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification
  6. 2025FDBPL: Faster Distillation-Based Prompt Learning for Region-Aware Vision-Language Models Adaptation
  7. 2024PromptSync: Bridging Domain Gaps in Vision-Language Models through Class-Aware Prototype Alignment and Discrimination
  8. 2024Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation