mlsanityData to 14 Oct 2026

Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Model

arXiv 2509.03895 ↗First posted 4 Sep 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Contrastive vision-language models excel in zero-shot image recognition but face challenges in few-shot scenarios due to computationally intensive offline fine-tuning using prompt learning, which risks overfitting. To overcome these limitations, we propose Attn-Adapter, a novel online few-shot learning framework that enhances CLIP's adaptability via a dual attention mechanism. Our design incorporates dataset-specific information through two components: the Memory Attn-Adapter, which refines category embeddings using support examples, and the Local-Global Attn-Adapter, which enriches image embeddings by integrating local and global features. This architecture enables dynamic adaptation from a few labeled samples without retraining the base model. Attn-Adapter outperforms state-of-the-art methods in cross-category and cross-dataset generalization, maintaining efficient inference and scaling across CLIP backbones.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 36 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

27 of 36 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Adaptive Cache Enhancement for Test-Time Adaptation of Vision-Language Models11 Aug
  2. 2025Accelerating Conditional Prompt Learning via Masked Image Modeling for Vision-Language Models7 Aug
  3. 2024GalLoP: Learning Global and Local Prompts for Vision-Language Models1 Jul
  4. 2023Meta-Adapter: An Online Few-shot Learner for Vision-Language Model7 Nov
  5. 2023Proto-CLIP: Vision-Language Prototypical Network for Few-Shot Learning6 Jul
  6. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  7. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  8. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  9. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  6. 2013Describing Textures in the Wildcited by 2,125
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  8. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  10. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  11. 2019Natural Adversarial Examplescited by 1,408
  12. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
6 more
  1. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  2. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  3. 2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
  4. 2023Boosting Semi-Supervised Learning by bridging high and low-confidence predictionscited by 11
  5. 2023SequenceMatch: Revisiting the design of weak-strong augmentations for Semi-supervised learningcited by 8
  6. 2023Debiasing, calibrating, and improving Semi-supervised Learning performance via simple Ensemble Projectorcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
  2. 2024Dual Memory Networks: A Versatile Adaptation Approach for Vision-Language Models
  3. 2026A3B2: Adaptive Asymmetric Adapter for Alleviating Branch Bias in Vision-Language Image Classification with Few-Shot Learning
  4. 2022Localized Latent Updates for Fine-Tuning Vision-Language Models
  5. 2024Multi-Modal Adapter for Vision-Language Models
  6. 2023A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models
  7. 2025ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models
  8. 2026Label Shift Aware Adaptation for Online Zero-shot Learning with Contrastive Language-Image Pre-Training (CLIP)