mlsanityData to 7 Oct 2026

What Makes Good Examples for Visual In-Context Learning?

arXiv 2301.13670 ↗First posted 31 Jan 2023cs.CVcited by 150

Not on the map. Why?

Cited by 31 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large-scale models trained on broad data have recently become the mainstream architecture in computer vision due to their strong generalization performance. In this paper, the main focus is on an emergent ability in large vision models, known as in-context learning, which allows inference on unseen tasks by conditioning on in-context examples (a.k.a.~prompt) without updating the model parameters. This concept has been well-known in natural language processing but has only been studied very recently for large vision models. We for the first time provide a comprehensive investigation on the impact of in-context examples in computer vision, and find that the performance is highly sensitive to the choice of in-context examples. To overcome the problem, we propose a prompt retrieval framework to automate the selection of in-context examples. Specifically, we present (1) an unsupervised prompt retrieval method based on nearest example search using an off-the-shelf model, and (2) a supervised prompt retrieval method, which trains a neural network to choose examples that directly maximize in-context learning performance. The results demonstrate that our methods can bring non-trivial improvements to visual in-context learning in comparison to the commonly-used random selection.

Key facts. Cited by 150 papers in the map (data through 7 Oct 2026). Builds on 28 of its 30 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

28 of 30 references.

References · 28

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
16 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  3. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  4. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  6. 2021Domain Generalization: A Surveycited by 760
  7. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  9. 2021MetaICL: Learning to Learn In Contextcited by 450
  10. 2017One-Shot Learning for Semantic Segmentationcited by 337
  11. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 317
  12. 2022Visual Prompting via Image Inpaintingcited by 248
  13. 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
  14. 2022In-context Examples Selection for Machine Translationcited by 172
  15. 2022Neural Prompt Searchcited by 133
  16. 2022Using natural language prompts for machine translationcited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 22; 2024: 54; 2025: 40; 2026: 34.

150 papers.

Sort:

By area:

  1. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 267
  2. 2024Visual In-Context Learning for Large Vision-Language Modelscited by 124
  3. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 124
  4. 2023In-Context Learning Unlocked for Diffusion Modelscited by 103
  5. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 85
  6. 2025OmniSVG: A Unified Scalable Vector Graphics Generation Modelcited by 71
  7. 2023ImageBrush: Learning Visual In-Context Instructions for Exemplar-Based Image Manipulationcited by 59
  8. 2024ControlVAR: Exploring Controllable Visual Autoregressive Modelingcited by 58
  9. 2023Towards In-context Scene Understandingcited by 45
  10. 2023Exploring Effective Factors for Improving Visual In-Context Learningcited by 43
  11. 2023CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Updatecited by 42
  12. 2023HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Modelscited by 41
38 more
  1. 2023Explore In-Context Learning for 3D Point Cloud Understandingcited by 40
  2. 2024BadRobot: Jailbreaking Embodied LLM Agents in the Physical Worldcited by 40
  3. 2025VisualCloze: A Universal Image Generation Framework via Visual In-Context Learningcited by 39
  4. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  5. 2023Visual Tuningcited by 34
  6. 2024VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learningcited by 34
  7. 2024What Factors Affect Multi-Modal In-Context Learning? An In-Depth Explorationcited by 31
  8. 2024Reason-before-Retrieve: One-Stage Reflective Chain-of-Thoughts for Training-Free Zero-Shot Composed Image Retrievalcited by 31
  9. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  10. 2023Instruct Me More! Random Prompting for Visual In-Context Learningcited by 30
  11. 2024On the Out-Of-Distribution Generalization of Multimodal Large Language Modelscited by 30
  12. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  13. 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 30
  14. 2023Unifying Image Processing as Visual Prompting Question Answeringcited by 29
  15. 2023Large Language Model for Multi-objective Evolutionary Optimizationcited by 29
  16. 2024Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimizationcited by 28
  17. 2023Visual Instruction Inversion: Image Editing via Visual Promptingcited by 26
  18. 2024Finding Visual Task Vectorscited by 24
  19. 2025Edit Transfer: Learning Image Editing via Vision In-Context Relationscited by 23
  20. 2023SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channelscited by 20
  21. 2023Context-Aware Meta-Learningcited by 20
  22. 2023Retrieval-augmented Multi-modal Chain-of-Thoughts Reasoning for Large Language Modelscited by 18
  23. 2024The Curious Case of Nonverbal Abstract Reasoning with Multi-Modal Large Language Modelscited by 17
  24. 2024VipAct: Visual-Perception Enhancement via Specialized VLM Agent Collaboration and Tool-usecited by 17
  25. 2023Cost-Effective In-Context Learning for Entity Resolution: A Design Space Explorationcited by 15
  26. 2024Towards Global Optimal Visual In-Context Learning Prompt Selectioncited by 15
  27. 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 15
  28. 2025Show and Segment: Universal Medical Image Segmentation via In-Context Learningcited by 15
  29. 2023Instruction-ViT: Multi-Modal Prompts for Instruction Learning in ViTcited by 14
  30. 2024VASCAR: Content-Aware Layout Generation via Visual-Aware Self-Correctioncited by 14
  31. 2024Low-Resource Vision Challenges for Foundation Modelscited by 13
  32. 2024Can MLLMs Perform Text-to-Image In-Context Learning?cited by 13
  33. 2024Exploring the Robustness of In-Context Learning with Noisy Labelscited by 13
  34. 2024Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoningcited by 13
  35. 2024DG-PIC: Domain Generalized Point-In-Context Learning for Point Cloud Understandingcited by 13
  36. 2024Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulationcited by 13
  37. 2024Explore In-Context Segmentation via Latent Diffusion Modelscited by 12
  38. 2024LLMs in Biomedicine: A study on clinical Named Entity Recognitioncited by 12

50 most cited of 150 All 150 (JSON)