mlsanityData to 7 Oct 2026

Fair Context Learning for Evidence-Balanced Test-Time Adaptation in Vision-Language Models

arXiv 2602.07027 ↗First posted 2 Feb 2026cs.CV cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Vision-Language Models (VLMs) such as CLIP enable strong zero-shot recognition but suffer substantial degradation under distribution shifts. Test-Time Adaptation (TTA) aims to improve robustness using only unlabeled test samples, yet most prompt-based TTA methods rely on entropy minimization -- an approach that can amplify spurious correlations and induce overconfident errors when classes share visual features. We propose Fair Context Learning (FCL), an episodic TTA framework that avoids entropy minimization by explicitly addressing shared-evidence bias. Motivated by our additive evidence decomposition assumption, FCL decouples adaptation into (i) augmentation-based exploration to identify plausible class candidates, and (ii) fairness-driven calibration that adapts text contexts to equalize sensitivity to common visual evidence. This fairness constraint mitigates partial feature obsession and enables effective calibration of text embeddings without relying on entropy reduction. Through extensive evaluation, we empirically validate our theoretical motivation and show that FCL achieves competitive adaptation performance relative to state-of-the-art TTA methods across diverse domain-shift and fine-grained benchmarks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 47 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

36 of 47 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models30 Jun
  2. 2024Test-Time Low Rank Adaptation via Confidence Maximization for Zero-Shot Generalization of Vision-Language Models22 Jul
  3. 2024Frustratingly Easy Test-Time Adaptation of Vision-Language Models28 May
  4. 2024On the test-time zero-shot generalization of vision-language models: Do we really need prompt learning?3 May
  5. 2024C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion21 Mar
  6. 2024Just Shift It: Test-Time Prototype Shifting for Zero-Shot Generalization with Vision-Language Models19 Mar
  7. 2023Align Your Prompts: Test-Time Prompting with Distribution Alignment for Zero-Shot Generalization2 Nov
  8. 2023Test-Time Adaptation with CLIP Reward for Zero-Shot Generalization in Vision-Language Models29 May
  9. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  10. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  11. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  6. 2013Describing Textures in the Wildcited by 2,110
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  8. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  11. 2019Natural Adversarial Examplescited by 1,401
  12. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
13 more
  1. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  2. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
  3. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  4. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  5. 2022Continual Test-Time Domain Adaptationcited by 520
  6. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  7. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  8. 2023Robust Test-Time Adaptation in Dynamic Scenarioscited by 185
  9. 2023Does CLIP's Generalization Performance Mainly Stem from High Train-Test Similarity?cited by 35
  10. 2025R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuningcited by 33
  11. 2022Feature Dropout: Revisiting the Role of Augmentations in Contrastive Learningcited by 9
  12. 2023Understanding Data Augmentation from a Robustness Perspectivecited by 3
  13. 2024AdaMER-CTC: Connectionist Temporal Classification with Adaptive Maximum Entropy Regularization for Automatic Speech Recognitioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Fairness-Aware Test-Time Prompt Tuning
  2. 2025Bayesian Test-Time Adaptation for Vision-Language Models
  3. 2026Ramen: Robust Test-Time Adaptation of Vision-Language Models with Active Sample Selection
  4. 2025ETTA: Efficient Test-Time Adaptation for Vision-Language Models through Dynamic Embedding Updates
  5. 2025Uniformity First: Uniformity-aware Test-time Adaptation of Vision-language Models against Image Corruption
  6. 2024CLIPArTT: Adaptation of CLIP to New Domains at Test Time
  7. 2025Advancing Reliable Test-Time Adaptation of Vision-Language Models under Visual Variations
  8. 2026ProtoDCS: Towards Robust and Efficient Open-Set Test-Time Adaptation for Vision-Language Models