mlsanityData to 7 Oct 2026

ECO: Ensembling Context Optimization for Vision-Language Models

arXiv 2307.14063 ↗First posted 26 Jul 2023cs.CVcited by 4

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Image recognition has recently witnessed a paradigm shift, where vision-language models are now used to perform few-shot classification based on textual prompts. Among these, the CLIP model has shown remarkable capabilities for zero-shot transfer by matching an image and a custom textual prompt in its latent space. This has paved the way for several works that focus on engineering or learning textual contexts for maximizing CLIP's classification capabilities. In this paper, we follow this trend by learning an ensemble of prompts for image classification. We show that learning diverse and possibly shorter contexts improves considerably and consistently the results rather than relying on a single trainable prompt. In particular, we report better few-shot capabilities with no additional cost at inference time. We demonstrate the capabilities of our approach on 11 different benchmarks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 12 of its 24 references that are in the map. First posted on arXiv 26 Jul 2023.

Builds on

12 of 24 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Conditional Prompt Learning for Vision-Language Models10 Mar
  2. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 10

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2013Describing Textures in the Wildcited by 2,110
  4. 2013Fine-Grained Visual Classification of Aircraftcited by 1,701
  5. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  6. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  7. 2022MaPLe: Multi-modal Prompt Learningcited by 685
  8. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  9. 2023Zero-Shot Composed Image Retrieval with Textual Inversioncited by 171
  10. 2021Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Searchcited by 57

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 3.

4 papers.

  1. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  2. 2024GalLoP: Learning Global and Local Prompts for Vision-Language Modelscited by 29
  3. 2024Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillationcited by 18
  4. 2024From Open Vocabulary to Open World: Teaching Vision Language Models to Detect Novel Objectscited by 5

Similar, not cited

Close in wording, no citation either way

  1. 2025Cluster-Aware Prompt Ensemble Learning for Few-Shot Vision-Language Model Adaptation
  2. 2026Beyond Heuristic Prompting: A Concept-Guided Bayesian Framework for Zero-Shot Image Recognition
  3. 2024CLIPArTT: Adaptation of CLIP to New Domains at Test Time
  4. 2026Low-Rank Prompt Learning for Vision-Language Models with Fixed-Token Bases
  5. 2022PLOT: Prompt Learning with Optimal Transport for Vision-Language Models
  6. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapters
  7. 2026AdaBoosting Text Prompts for Vision-Language Models
  8. 2023Towards Difficulty-Agnostic Efficient Transfer Learning for Vision-Language Models