mlsanityData to 14 Oct 2026

Flamingo: a Visual Language Model for Few-Shot Learning

arXiv 2204.14198 ↗First posted 29 Apr 2022cs.CV cs.AI cs.LGcited by 4,386

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research. We introduce Flamingo, a family of Visual Language Models (VLM) with this ability. We propose key architectural innovations to: (i) bridge powerful pretrained vision-only and language-only models, (ii) handle sequences of arbitrarily interleaved visual and textual data, and (iii) seamlessly ingest images or videos as inputs. Thanks to their flexibility, Flamingo models can be trained on large-scale multimodal web corpora containing arbitrarily interleaved text and images, which is key to endow them with in-context few-shot learning capabilities. We perform a thorough evaluation of our models, exploring and measuring their ability to rapidly adapt to a variety of image and video tasks. These include open-ended tasks such as visual question-answering, where the model is prompted with a question which it has to answer; captioning tasks, which evaluate the ability to describe a scene or an event; and close-ended tasks such as multiple-choice visual question-answering. For tasks lying anywhere on this spectrum, a single Flamingo model can achieve a new state of the art with few-shot learning, simply by prompting the model with task-specific examples. On numerous benchmarks, Flamingo outperforms models fine-tuned on thousands of times more task-specific data.

Key facts. Cited by 4,386 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 138 of its 184 references that are in the map. First posted on arXiv 29 Apr 2022.

Builds on

138 of 184 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation12 Mar
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan
  3. 2021Scaling Up Vision-Language Pre-training for Image Captioning24 Nov
  4. 2021FILIP: Fine-grained Interactive Language-Image Pre-Training9 Nov
  5. 2021VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts3 Nov
  6. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervision24 Aug

Other references · 132

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  9. 2020End-to-End Object Detection with Transformerscited by 7,414
  10. 2020Scaling Laws for Neural Language Modelscited by 6,586
  11. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
38 more
  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  8. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  9. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  10. 2015VQA: Visual Question Answeringcited by 2,506
  11. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  12. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  13. 2016Matching Networks for One Shot Learningcited by 2,255
  14. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  15. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  16. 2019Towards VQA Models That Can Readcited by 1,718
  17. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  18. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  19. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  20. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  21. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  22. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  23. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  24. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  25. 2018Datasheets for Datasetscited by 1,386
  26. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  27. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  28. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  29. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  30. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  31. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  32. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  33. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  34. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,121
  35. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  36. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  37. 2018Model Cards for Model Reportingcited by 1,080
  38. 2022Red Teaming Language Models with Language Modelscited by 1,026

50 most cited of 132 All 132 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 183; 2023: 870; 2024: 1222; 2025: 1207; 2026: 904.

4,386 papers.

Sort:

By area:

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  10. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  11. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  12. 2022Emergent Abilities of Large Language Modelscited by 2,512
38 more
  1. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  7. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  8. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  10. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  11. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  12. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  13. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  14. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  15. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  16. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  17. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  18. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  19. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  20. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  21. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  22. 2022Editing Models with Task Arithmeticcited by 1,120
  23. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  24. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  25. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,081
  26. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
  27. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  28. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  29. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  30. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  31. 2023A Survey on Multimodal Large Language Modelscited by 909
  32. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  33. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  34. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  35. 2022A Generalist Agentcited by 808
  36. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  37. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 781
  38. 2024Emu3: Next-Token Prediction is All You Needcited by 781

50 most cited of 4,386 All 4,386 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Coling-UniA at SciVQA 2025: Few-Shot Example Retrieval and Confidence-Informed Ensembling for Multimodal Large Language Models
  2. 2023InfMLLM: A Unified Framework for Visual-Language Tasks
  3. 2026FewMMBench: A Benchmark for Multimodal Few-Shot Learning
  4. 2025How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding
  5. 2023Pushing Boundaries: Exploring Zero Shot Object Classification with Large Multimodal Models
  6. 2024A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasets
  7. 2025VisualQuest: A Benchmark for Abstract Visual Reasoning in MLLMs
  8. 2024VisualRWKV: Exploring Recurrent Neural Networks for Visual Language Models