mlsanityData to 7 Oct 2026

UniAdapter: Unified Parameter-Efficient Transfer Learning for Cross-modal Modeling

arXiv 2302.06605 ↗First posted 13 Feb 2023cs.CV cs.CLcited by 40

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Large-scale vision-language pre-trained models have shown promising transferability to various downstream tasks. As the size of these foundation models and the number of downstream tasks grow, the standard full fine-tuning paradigm becomes unsustainable due to heavy computational and storage costs. This paper proposes UniAdapter, which unifies unimodal and multimodal adapters for parameter-efficient cross-modal adaptation on pre-trained vision-language models. Specifically, adapters are distributed to different modalities and their interactions, with the total number of tunable parameters reduced by partial weight sharing. The unified and knowledge-sharing design enables powerful cross-modal representations that can benefit various downstream tasks, requiring only 1.0%-2.0% tunable parameters of the pre-trained model. Extensive experiments on 6 cross-modal downstream benchmarks (including video-text retrieval, image-text retrieval, VideoQA, and VQA) show that in most cases, UniAdapter not only outperforms the state-of-the-arts, but even beats the full fine-tuning strategy. Particularly, on the MSRVTT retrieval task, UniAdapter achieves 49.7% recall@1 with 2.2% model parameters, outperforming the latest competitors by 2.0%. The code and models are available at https://github.com/RERV/UniAdapter.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 40 of its 48 references that are in the map. First posted on arXiv 13 Feb 2023.

Builds on

40 of 48 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  12. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
28 more
  1. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  3. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  4. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  5. 2021ViLT: Vision-and-Language Transformer Without Convolution or Region Supervisioncited by 1,224
  6. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  8. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  9. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  10. 2017Localizing Moments in Video with Natural Languagecited by 660
  11. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 645
  12. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  13. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  14. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  15. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  16. 2020Large-Scale Adversarial Training for Vision-and-Language Representation Learningcited by 327
  17. 2021VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Taskscited by 319
  18. 2021MERLOT: Multimodal Neural Script Knowledge Modelscited by 311
  19. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 309
  20. 2020Just Ask: Learning to Answer Questions from Millions of Narrated Videoscited by 248
  21. 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
  22. 2021Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learningcited by 187
  23. 2021VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modelingcited by 187
  24. 2022All in One: Exploring Unified Video-Language Pre-trainingcited by 171
  25. 2021Align and Prompt: Video-and-Language Pre-training with Entity Promptscited by 149
  26. 2022OmniVL:One Foundation Model for Image-Language and Video-Language Taskscited by 143
  27. 2022A CLIP-Hitchhiker's Guide to Long Video Retrievalcited by 57
  28. 2022LGDN: Language-Guided Denoising Network for Video-Language Modelingcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 12; 2025: 16; 2026: 4.

40 papers.

Sort:

By area:

  1. 2023Unified Coarse-to-Fine Alignment for Video-Text Retrievalcited by 49
  2. 2024SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentationcited by 43
  3. 2023Visual Tuningcited by 34
  4. 2024Few-shot Adaptation of Multi-modal Foundation Models: A Surveycited by 28
  5. 2023UniPT: Universal Parallel Tuning for Transfer Learning with Efficient Parameter and Memorycited by 26
  6. 2023Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrievalcited by 24
  7. 2023Parameter-efficient Tuning of Large-scale Multimodal Foundation Modelcited by 23
  8. 2023VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Controlcited by 21
  9. 2024Memory-Space Visual Prompting for Efficient Vision-Language Fine-Tuningcited by 20
  10. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Roboticscited by 17
  11. 2024RAP: Efficient Text-Video Retrieval with Sparse-and-Correlated Adaptercited by 16
  12. 2023Side4Video: Spatial-Temporal Side Network for Memory-Efficient Image-to-Video Transfer Learningcited by 11
28 more
  1. 2024SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learningcited by 11
  2. 2025SwimVG: Step-wise Multimodal Fusion and Adaption for Visual Groundingcited by 8
  3. 2024TokenBinder: Text-Video Retrieval with One-to-Many Alignment Paradigmcited by 6
  4. 2025Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigationcited by 5
  5. 2025Faster Parameter-Efficient Tuning with Token Redundancy Reductioncited by 4
  6. 2025Global Minimizers of Sigmoid Contrastive Losscited by 4
  7. 2025DitHub: A Modular Framework for Incremental Open-Vocabulary Object Detectioncited by 3
  8. 2025Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instructioncited by 3
  9. 2024Efficient Remote Sensing with Harmonized Transfer Learning and Modality Alignmentcited by 2
  10. 2023When Parameter-efficient Tuning Meets General-purpose Vision-language Modelscited by 1
  11. 2024Harnessing Vision-Language Pretrained Models with Temporal-Aware Adaptation for Referring Video Object Segmentationcited by 1
  12. 2024Beyond Coarse-Grained Matching in Video-Text Retrievalcited by 1
  13. 2025Representation Discrepancy Bridging Method for Remote Sensing Image-Text Retrievalcited by 1
  14. 2025Dynamic Embedding of Hierarchical Visual Features for Efficient Vision-Language Fine-Tuningcited by 1
  15. 2025Enhancing Semantic Segmentation with Continual Self-Supervised Pre-trainingcited by 1
  16. 2025TCMA: Text-Conditioned Multi-granularity Alignment for Drone Cross-Modal Text-Video Retrievalcited by 1
  17. 2025Null-LoRA: Low-Rank Adaptation on Null Spacecited by 1
  18. 2024Deployment Prior Injection for Run-time Calibratable Object Detectioncited by 0
  19. 2024ContextBLIP: Doubly Contextual Alignment for Contrastive Image Retrieval from Linguistically Complex Descriptionscited by 0
  20. 2025A Comprehensive Survey of Challenges and Opportunities of Few-Shot Learning Across Multiple Domainscited by 0
  21. 2025LARGO: Low-Rank Regulated Gradient Projection for Robust Parameter Efficient Fine-Tuningcited by 0
  22. 2025Gradient-based Fine-Tuning through Pre-trained Model Regularizationcited by 0
  23. 2025Remodeling Semantic Relationships in Vision-Language Fine-Tuningcited by 0
  24. 2025Hierarchy-Aware Fine-Tuning of Vision-Language Modelscited by 0
  25. 2026Multi-Perspective Subimage CLIP with Keyword Guidance for Remote Sensing Image-Text Retrievalcited by 0
  26. 2026D2-LoRA: A Synergistic Approach to Differential and Directional Low-Rank Adaptationcited by 0
  27. 2026AgriField-40K: Adapting Vision Models to Agriculture With Efficient Continual Pretrainingcited by 0
  28. 2026Event Interaction in Low-Rank Bottlenecks for Temporal Relation Extractioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Cross-Modal Adapter: Parameter-Efficient Transfer Learning Approach for Vision-Language Models
  2. 2025RMAdapter: Reconstruction-based Multi-Modal Adapter for Vision-Language Models
  3. 2022Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks
  4. 2024CROME: Cross-Modal Adapters for Efficient Multimodal LLM
  5. 2024A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter
  6. 2026One Modality to Forget Them All: Enhancing Cross-Modal Unlearning in Vision-Language Models
  7. 2024Unsupervised Domain Adaption Harnessing Vision-Language Pre-training
  8. 2025Transferable Model-agnostic Vision-Language Model Adaptation for Efficient Weak-to-Strong Generalization