mlsanityData to 7 Oct 2026

A3B2: Adaptive Asymmetric Adapter for Alleviating Branch Bias in Vision-Language Image Classification with Few-Shot Learning

arXiv 2605.13161 ↗First posted 13 May 2026cs.CV cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Efficient transfer learning methods for large-scale vision-language models (e.g., CLIP) enable strong few-shot transfer, yet existing adaptation methods follow a fixed fine-tuning paradigm that implicitly assumes a uniform importance of the image and text branches, which has not been systematically studied in image classification. Through extensive analysis, we reveal a Branch Bias issue in vision-language image classification: adapting the image encoder does not always improve performance under out-of-distribution settings. Motivated by this observation, we propose A3B2, an Adaptive Asymmetric Adapter that alleviates Branch Bias in few-shot learning. A3B2 introduces Uncertainty-Aware Adapter Dampening (UAAD), which automatically suppresses image-branch adaptation when prediction uncertainty is high, enabling soft and data-driven control without manual intervention. Architecturally, A3B2 adopts a lightweight asymmetric design inspired by mixture-of-experts with Load Balancing Regularization. Extensive experiments on three few-shot image classification tasks across 11 datasets demonstrate that A3B2 consistently outperforms 11 competitive prompt- and adapter-based baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 45 of its 64 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

45 of 64 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024Vision-Language Model Fine-Tuning via Simple Parameter-Efficient Modification25 Sep
  2. 2023TCP:Textual-based Class-aware Prompt tuning for Visual-Language Model30 Nov
  3. 2023Read-only Prompt Optimization for Vision-Language Few-shot Learning29 Aug
  4. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimization23 Mar
  5. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentation23 Feb
  6. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  7. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters9 Oct
  8. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  7. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  8. 2013Describing Textures in the Wildcited by 2,110
  9. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  10. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  11. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  12. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
25 more
  1. 2019Natural Adversarial Examplescited by 1,401
  2. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  3. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  4. 2022Language-driven Semantic Segmentationcited by 711
  5. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  6. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 560
  7. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  8. 2024HydraLoRA: An Asymmetric LoRA Architecture for Efficient Fine-Tuningcited by 118
  9. 2024Higher Layers Need More LoRA Expertscited by 101
  10. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  11. 2021Go Wider Instead of Deepercited by 56
  12. 2025MMRL: Multi-Modal Representation Learning for Vision-Language Modelscited by 38
  13. 2025Disentangled Knowledge Tracing for Alleviating Cognitive Biascited by 12
  14. 2025Kernel-based Unsupervised Embedding Alignment for Enhanced Visual Representation in Vision-language Modelscited by 12
  15. 2025MoRE: A Mixture of Low-Rank Experts for Adaptive Multi-Task Learningcited by 9
  16. 2024MergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalitiescited by 6
  17. 2025PureKV: Plug-and-Play KV Cache Optimization with Spatial-Temporal Sparse Attention for Vision-Language Large Modelscited by 6
  18. 2025FlowMM: Cross-Modal Information Flow Guided KV Cache Merging for Efficient Multimodal Context Inferencecited by 6
  19. 2025AccKV: Towards Efficient Audio-Video LLMs Inference via Adaptive-Focusing and Cross-Calibration KV Cache Optimizationcited by 6
  20. 2025CoLA: Collaborative Low-Rank Adaptationcited by 5
  21. 2025Cuff-KT: Tackling Learners' Real-time Learning Pattern Adjustment via Tuning-Free Knowledge State Guided Model Updatingcited by 5
  22. 2025MMRL++: Parameter-Efficient and Interaction-Aware Representation Learning for Vision-Language Modelscited by 4
  23. 2025Language-Image Alignment with Fixed Text Encoderscited by 4
  24. 2025Multimodal Graph-Based Variational Mixture of Experts Network for Zero-Shot Multimodal Information Extractioncited by 3
  25. 2026Beyond Student: An Asymmetric Network for Neural Network Inheritancecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Attn-Adapter: Attention Is All You Need for Online Few-shot Learner of Vision-Language Model
  2. 2023A Closer Look at the Few-Shot Adaptation of Large Vision-Language Models
  3. 2023Meta-Adapter: An Online Few-shot Learner for Vision-Language Model
  4. 2025Rethinking Few-Shot Adaptation of Vision-Language Models in Two Stages
  5. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportation
  6. 2022Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
  7. 2026From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model
  8. 2024Enhancing Vision-Language Few-Shot Adaptation with Negative Learning