mlsanityData to 7 Oct 2026

Domain Recentering and Confidence-Weighted Prior Calibration for Vision-Language Models

arXiv 2609.29358 ↗First posted 24 Sep 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Vision-language models such as CLIP achieve strong zero-shot classification, yet under distribution shift, visual embeddings drift from fixed text embeddings. Training-free calibration avoids the per-sample optimization of prompt learning, but prior feature calibration gives each image the full bias of one hard cluster. We propose Domain Recentering with Confidence Calibration (DRC), a training-free method adapting CLIP from a set of unlabeled target images. DRC fits a Gaussian mixture once and subtracts from each embedding a posterior-weighted average of component means. It then removes residual class preference with a log-prior correction, estimating the prior from confidence-weighted predictions. Among compared methods, DRC achieves the highest average accuracy on cross-domain datasets, exceeding zero-shot CLIP by 4.13 and 5.07 points with ViT-B/16 and ResNet-50, with gains over CLIP also holding under ImageNet distribution shifts.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 13 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

12 of 13 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024UMFC: Unsupervised Multi-Domain Feature Calibration for Vision-Language Models11 Nov
  2. 2024Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correcting25 Oct
  3. 2023Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning11 Aug
  4. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  5. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  6. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 6

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  3. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  4. 2019Natural Adversarial Examplescited by 1,401
  5. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  6. 2024The Neglected Tails in Vision-Language Modelscited by 63

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards Calibrating Prompt Tuning of Vision-Language Models
  2. 2025Latent Domain Prompt Learning for Vision-Language Models
  3. 2023Enabling Calibration In The Zero-Shot Inference of Large Vision-Language Models
  4. 2024CLIP the Divergence: Language-guided Unsupervised Domain Adaptation
  5. 2023ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation
  6. 2025Confidence-calibrated covariate shift correction for few-shot classification in Vision-Language Models
  7. 2024BaFTA: Backprop-Free Test-Time Adaptation For Zero-Shot Vision-Language Models
  8. 2023Robust Fine-Tuning of Vision-Language Models for Domain Generalization