mlsanityData to 14 Oct 2026

Meta-Adapter: An Online Few-shot Learner for Vision-Language Model

arXiv 2311.03774 ↗First posted 7 Nov 2023cs.CVcited by 31

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

The contrastive vision-language pre-training, known as CLIP, demonstrates remarkable potential in perceiving open-world visual concepts, enabling effective zero-shot image recognition. Nevertheless, few-shot learning methods based on CLIP typically require offline fine-tuning of the parameters on few-shot samples, resulting in longer inference time and the risk of over-fitting in certain domains. To tackle these challenges, we propose the Meta-Adapter, a lightweight residual-style adapter, to refine the CLIP features guided by the few-shot samples in an online manner. With a few training samples, our method can enable effective few-shot learning capabilities and generalize to unseen data or tasks without additional fine-tuning, achieving competitive performance and high efficiency. Without bells and whistles, our approach outperforms the state-of-the-art online few-shot learning method by an average of 3.6% on eight image classification datasets with higher inference speed. Furthermore, our model is simple and flexible, serving as a plug-and-play module directly applicable to downstream tasks. Without further fine-tuning, Meta-Adapter obtains notable performance improvements in open-vocabulary object detection and segmentation tasks.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 56 of its 70 references that are in the map. First posted on arXiv 7 Nov 2023.

Builds on

56 of 70 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models4 Nov
  2. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  3. 2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling6 Nov
  4. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  5. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  8. 2017Mask R-CNNcited by 7,277
  9. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  11. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  12. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
38 more
  1. 2017Non-local Neural Networkscited by 2,293
  2. 2013Describing Textures in the Wildcited by 2,125
  3. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  4. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  5. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  6. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  7. 2019Natural Adversarial Examplescited by 1,408
  8. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  9. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  10. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  11. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  12. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  13. 2019How Can We Know What Language Models Know?cited by 959
  14. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  15. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  16. 2021Robust fine-tuning of zero-shot modelscited by 797
  17. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  18. 2016Learning to learn by gradient descent by gradient descentcited by 620
  19. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  20. 2021Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigmcited by 405
  21. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  22. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  23. 2023Personalize Segment Anything Model with One Shotcited by 250
  24. 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
  25. 2023Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learnerscited by 152
  26. 2021Aligning Pretraining for Detection via Object-Level Contrastive Learningcited by 110
  27. 2023Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinementcited by 84
  28. 2020Learning Dynamic Routing for Semantic Segmentationcited by 79
  29. 2020End-to-End Object Detection with Fully Convolutional Networkcited by 70
  30. 2019TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detectioncited by 36
  31. 2022PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Modelscited by 30
  32. 2019Learnable Tree Filter for Structure-preserving Feature Transformcited by 25
  33. 2022DBQ-SSD: Dynamic Ball Query for Efficient 3D Object Detectioncited by 25
  34. 2023Dynamic Grained Encoder for Vision Transformerscited by 24
  35. 2020Fine-Grained Dynamic Head for Object Detectioncited by 16
  36. 2023BoxSnake: Polygonal Instance Segmentation with Box Supervisioncited by 12
  37. 2020Rethinking Learnable Tree Filter for Generic Feature Transformcited by 10
  38. 2021Workshop on Autonomous Driving at CVPR 2021: Technical Report for Streaming Perception Challengecited by 6

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 19; 2026: 1.

31 papers.

Sort:

By area:

  1. 2025A Survey on Efficient Vision-Language Modelscited by 19
  2. 2024Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentationcited by 17
  3. 2025Keeping Yourself is Important in Downstream Tuning Multimodal Large Language Modelcited by 16
  4. 2024GrootVL: Tree Topology is All You Need in State Space Modelcited by 13
  5. 2024VLSM-Adapter: Finetuning Vision-Language Segmentation Efficiently with Lightweight Blockscited by 11
  6. 2025SANSA: Unleashing the Hidden Semantics in SAM2 for Few-Shot Segmentationcited by 8
  7. 2025Advancing Multimodal In-Context Learning in Large Vision-Language Models with Task-aware Demonstrationscited by 7
  8. 2024PromptTA: Prompt-driven Text Adapter for Source-free Domain Generalizationcited by 6
  9. 2025Diff-Prompt: Diffusion-Driven Prompt Generator with Mask Supervisioncited by 3
  10. 2025From Prediction to Perfection: Introducing Refinement to Autoregressive Image Generationcited by 3
  11. 2024Cross-Modal Few-Shot Learning: a Generative Transfer Learning Frameworkcited by 2
  12. 2024Data-Efficient CLIP-Powered Dual-Branch Networks for Source-Free Unsupervised Domain Adaptationcited by 2
19 more
  1. 2024AdaptGCD: Multi-Expert Adapter Tuning for Generalized Category Discoverycited by 2
  2. 2024Test-time Alignment-Enhanced Adapter for Vision-Language Modelscited by 2
  3. 2025Mamba-Adaptor: State Space Model Adaptor for Visual Recognitioncited by 2
  4. 2025Enabling Validation for Robust Few-Shot Recognitioncited by 2
  5. 2024LLM meets Vision-Language Models for Zero-Shot One-Class Classificationcited by 1
  6. 2024CLIP's Visual Embedding Projector is a Few-shot Cornucopiacited by 1
  7. 2025LangDA: Building Context-Awareness via Language for Domain Adaptive Semantic Segmentationcited by 1
  8. 2025MSGCoOp: Multiple Semantic-Guided Context Optimization for Few-Shot Learningcited by 1
  9. 2025Delta Energy: Optimizing Energy Change During Vision-Language Alignment Improves both OOD Detection and OOD Generalizationcited by 1
  10. 2024Enhancing Robustness of Vision-Language Models through Orthogonality Learning and Self-Regularizationcited by 0
  11. 2025Adversarial Robustness for Unified Multi-Modal Encoders via Efficient Calibrationcited by 0
  12. 2025Your Offline Policy is Not Trustworthy: Bilevel Reinforcement Learning for Sequential Portfolio Optimizationcited by 0
  13. 2025Adapting Foundation Vision-Language Models to Medical Diagnosis via Query-Driven Expert Bridgingcited by 0
  14. 2025Optimizing Active Learning in Vision-Language Models via Parameter-Efficient Uncertainty Calibrationcited by 0
  15. 2025Color as the Impetus: Transforming Few-Shot Learnercited by 0
  16. 2025Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Modelcited by 0
  17. 2025Self Identity Mappingcited by 0
  18. 2025Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctorscited by 0
  19. 2026Domain Generalizable Adaptation of 3D Vision-Language Models via Regularized Fine-Tuningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
  2. 2022SVL-Adapter: Self-Supervised Adapter for Vision-Language Pretrained Models
  3. 2022SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classification
  4. 2024Low-Rank Few-Shot Adaptation of Vision-Language Models
  5. 2024Enhancing Vision-Language Few-Shot Adaptation with Negative Learning
  6. 2024Making Large Vision Language Models to be Good Few-shot Learners
  7. 2024Multi-Modal Adapter for Vision-Language Models
  8. 2022Localized Latent Updates for Fine-Tuning Vision-Language Models