mlsanityData to 7 Oct 2026

MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting

arXiv 2210.07179 ↗First posted 13 Oct 2022cs.CV cs.AI cs.CLcited by 41

Not on the map. Why?

Abstract (arXiv)

Large pre-trained models have proved to be remarkable zero- and (prompt-based) few-shot learners in unimodal vision and language tasks. We propose MAPL, a simple and parameter-efficient method that reuses frozen pre-trained unimodal models and leverages their strong generalization capabilities in multimodal vision-language (VL) settings. MAPL learns a lightweight mapping between the representation spaces of unimodal models using aligned image-text data, and can generalize to unseen VL tasks from just a few in-context examples. The small number of trainable parameters makes MAPL effective at low-data and in-domain learning. Moreover, MAPL's modularity enables easy extension to other pre-trained models. Extensive experiments on several visual question answering and image captioning benchmarks show that MAPL achieves superior or competitive performance compared to similar methods while training orders of magnitude fewer parameters. MAPL can be trained in just a few hours using modest computational resources and public datasets. We release our code and pre-trained model weights at https://github.com/mair-lab/mapl.

Key facts. Cited by 41 papers in the map (data through 7 Oct 2026). Builds on 39 of its 50 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

39 of 50 references.

References · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  11. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  12. 2019Towards VQA Models That Can Readcited by 1,696
27 more
  1. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  2. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  3. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  4. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  5. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  6. 2018Self-Attention with Relative Position Representationscited by 1,112
  7. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  8. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  9. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  10. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  11. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  12. 2021ClipCap: CLIP Prefix for Image Captioningcited by 557
  13. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
  14. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  15. 2020TextCaps: a Dataset for Image Captioning with Reading Comprehensioncited by 398
  16. 2016Visual Dialogcited by 377
  17. 2021An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQAcited by 298
  18. 2019UNITER: UNiversal Image-TExt Representation Learningcited by 225
  19. 2016Professor Forcing: A New Algorithm for Training Recurrent Networkscited by 214
  20. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 190
  21. 2017Deep Learning is Robust to Massive Label Noisecited by 164
  22. 2022Linearly Mapping from Image to Text Spacecited by 139
  23. 2021A Good Prompt Is Worth Millions of Parameters: Low-resource Prompt-based Learning for Vision-Language Modelscited by 101
  24. 2022Language Models are General-Purpose Interfacescited by 96
  25. 2021MAGMA -- Multimodal Augmentation of Generative Models through Adapter-based Finetuningcited by 85
  26. 2022Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillationcited by 70
  27. 2022Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalizationcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 13; 2025: 9; 2026: 4.

41 papers.

Sort:

By area:

  1. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  2. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  3. 2024What matters when building vision-language models?cited by 321
  4. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  5. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 136
  6. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  7. 2023X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoningcited by 78
  8. 2023InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructionscited by 64
  9. 2023COLA: A Benchmark for Compositional Text-to-image Retrievalcited by 53
  10. 2023UnIVAL: Unified Model for Image, Video, Audio and Language Taskscited by 44
  11. 2023Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understandingcited by 40
  12. 2024A Concept-Based Explainability Framework for Large Multimodal Modelscited by 39
29 more
  1. 2023Efficient Multimodal Fusion via Interactive Promptingcited by 32
  2. 2023Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Modelscited by 27
  3. 2024Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputscited by 22
  4. 2023VL-PET: Vision-and-Language Parameter-Efficient Tuning via Granularity Controlcited by 21
  5. 2023Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Promptscited by 19
  6. 2024SQ-LLaVA: Self-Questioning for Large Vision-Language Assistantcited by 18
  7. 2024Improved Baselines for Data-efficient Perceptual Augmentation of LLMscited by 18
  8. 2025Analyzing Finetuning Representation Shift for Multimodal LLMs Steeringcited by 15
  9. 2023Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answeringcited by 14
  10. 2025Learning to Steer: Input-dependent Steering for Multimodal LLMscited by 13
  11. 2025CTRL-O: Language-Controllable Object-Centric Visual Representation Learningcited by 12
  12. 2025FREE: Fast and Robust Vision Language Models with Early Exitscited by 12
  13. 2024Why do LLaVA Vision-Language Models Reply to Images in English?cited by 11
  14. 2024PosSAM: Panoptic Open-vocabulary Segment Anythingcited by 9
  15. 2025Decoupled Global-Local Alignment for Improving Compositional Understandingcited by 9
  16. 2026LatentLens: Revealing Highly Interpretable Visual Tokens in LLMscited by 9
  17. 2023C3Net: Compound Conditioned ControlNet for Multimodal Content Generationcited by 8
  18. 2024Skipping Computations in Multimodal LLMscited by 7
  19. 2025Vision-Speech Models: Teaching Speech Models to Converse about Imagescited by 4
  20. 2024Diversify, Rationalize, and Combine: Ensembling Multiple QA Strategies for Zero-shot Knowledge-based VQAcited by 3
  21. 2023M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarioscited by 2
  22. 2023GPT4Image: Large Pre-trained Models Help Vision Models Learn Better on Perception Taskcited by 2
  23. 2024Translating speech with just imagescited by 2
  24. 2024Describing Images textitFast and Slow: Quantifying and Predicting the Variation in Human Signals during Visuo-Linguistic Processescited by 1
  25. 2025Multilingual Vision-Language Models, A Surveycited by 1
  26. 2025RORPCap: Retrieval-based Objects and Relations Prompt for Image Captioningcited by 0
  27. 2026The Hidden Evolution of Disguised Visual Context inside the VLMcited by 0
  28. 2026Dynamic Context Adapters: Efficiently Infusing History into Vision-and-Language Modelscited by 0
  29. 2026Using OCR Heads to Verbalize Image Semanticscited by 0