mlsanityData to 7 Oct 2026

Test-time Alignment-Enhanced Adapter for Vision-Language Models

arXiv 2411.15735 ↗First posted 24 Nov 2024cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Test-time adaptation with pre-trained vision-language models (VLMs) has attracted increasing attention for tackling the issue of distribution shift during the test phase. While prior methods have shown effectiveness in addressing distribution shift by adjusting classification logits, they are not optimal due to keeping text features unchanged. To address this issue, we introduce a new approach called Test-time Alignment-Enhanced Adapter (TAEA), which trains an adapter with test samples to adjust text features during the test phase. We can enhance the text-to-image alignment prediction by utilizing an adapter to adapt text features. Furthermore, we also propose to adopt the negative cache from TDA as enhancement module, which further improves the performance of TAEA. Our approach outperforms the state-of-the-art TTA method of pre-trained VLMs by an average of 0.75% on the out-of-distribution benchmark and 2.5% on the cross-domain benchmark, with an acceptable training time. Code will be available at https://github.com/BaoshunWq/clip-TAEA.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 39 references that are in the map. First posted on arXiv 24 Nov 2024.

Builds on

25 of 39 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Efficient Test-Time Adaptation of Vision-Language Models27 Mar
  2. 2023Meta-Adapter: An Online Few-shot Learner for Vision-Language Model7 Nov
  3. 2023Diverse Data Augmentation with Diffusions for Effective Test-time Prompt Tuning11 Aug
  4. 2022Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models15 Sep
  5. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification19 Jul
  6. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  7. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  8. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2017Non-local Neural Networkscited by 2,290
  6. 2013Describing Textures in the Wildcited by 2,110
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  8. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  11. 2019Natural Adversarial Examplescited by 1,401
  12. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
5 more
  1. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  2. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  3. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  4. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  5. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Adapting Vision-Language Models Without Labels: A Comprehensive Surveycited by 6
  2. 2025ATAC: Augmentation-Based Test-Time Adversarial Correction for CLIPcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Prototype-Based Test-Time Adaptation of Vision-Language Models
  2. 2026Efficient Test-time Adaptation through Candidate Verification and Divergence Shifts
  3. 2025The Illusion of Progress? A Critical Look at Test-Time Adaptation for Vision-Language Models
  4. 2025Realistic Test-Time Adaptation of Vision-Language Models
  5. 2025Flatness Guided Test-Time Adaptation for Vision-Language Models
  6. 2024DOTA: Distributional Test-Time Adaptation of Vision-Language Models
  7. 2025Unleashing the Potential of All Test Samples: Mean-Shift Guided Test-Time Adaptation
  8. 2026Subspace Alignment for Vision-Language Model Test-time Adaptation