mlsanityData to 7 Oct 2026

Prompting Large Pre-trained Vision-Language Models For Compositional Concept Learning

arXiv 2211.05077 ↗First posted 9 Nov 2022cs.CVcited by 12

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

This work explores the zero-shot compositional learning ability of large pre-trained vision-language models(VLMs) within the prompt-based learning framework and propose a model (textitPromptCompVL) to solve the compositonal zero-shot learning (CZSL) problem. textitPromptCompVL makes two design choices: first, it uses a soft-prompting instead of hard-prompting to inject learnable parameters to reprogram VLMs for compositional learning. Second, to address the compositional challenge, it uses the soft-embedding layer to learn primitive concepts in different combinations. By combining both soft-embedding and soft-prompting, textitPromptCompVL achieves state-of-the-art performance on the MIT-States dataset. Furthermore, our proposed model achieves consistent improvement compared to other CLIP-based methods which shows the effectiveness of the proposed prompting strategies for CZSL.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 17 references that are in the map. First posted on arXiv 9 Nov 2022.

Builds on

13 of 17 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Learning to Compose Soft Prompts for Compositional Zero-Shot Learning7 Apr
  2. 2022Conditional Prompt Learning for Vision-Language Models10 Mar

Other references · 11

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  6. 2021GPT Understands, Toocited by 715
  7. 2019Task-Driven Modular Networks for Zero-Shot Compositional Learningcited by 121
  8. 2021Learning Graph Embeddings for Compositional Zero-shot Learningcited by 112
  9. 2021Open World Compositional Zero-Shot Learningcited by 100
  10. 2020Symmetry and Group in Attribute-Object Compositionscited by 89
  11. 2018Attributes as Operators: Factorizing Unseen Attribute-Object Compositionscited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 2; 2025: 4; 2026: 1.

12 papers.

  1. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 91
  2. 2023Troika: Multi-Path Cross-Modal Traction for Compositional Zero-Shot Learningcited by 26
  3. 2023Prompting Language-Informed Distribution for Compositional Zero-Shot Learningcited by 24
  4. 2023GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learningcited by 16
  5. 2025Learning Clustering-based Prototypes for Compositional Zero-shot Learningcited by 13
  6. 2023Compositional Zero-shot Learning via Progressive Language-based Observationscited by 12
  7. 2024TsCA: On the Semantic Consistency Alignment via Conditional Transport for Compositional Zero-Shot Learningcited by 6
  8. 2024Swing-by Dynamics in Concept Learning and Compositional Generalizationcited by 5
  9. 2025Learning Visual Proxy for Compositional Zero-Shot Learningcited by 2
  10. 2025Semantically Guided Dynamic Visual Prototype Refinement for Compositional Zero-Shot Learningcited by 0
  11. 2025EVA: Mixture-of-Experts Semantic Variant Alignment for Compositional Zero-Shot Learningcited by 0
  12. 2026Compositionality Emerges in a Narrow Depth-Connectivity Regime: Architecture Constraints and Solution Manifoldscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Decomposed Soft Prompt Guided Fusion Enhancing for Compositional Zero-Shot Learning
  2. 2025Visual Adaptive Prompting for Compositional Zero-Shot Learning
  3. 2023CAILA: Concept-Aware Intra-Layer Adapters for Compositional Zero-Shot Learning
  4. 2023DRPT: Disentangled and Recurrent Prompt Tuning for Compositional Zero-Shot Learning
  5. 2025Separated Inter/Intra-Modal Fusion Prompts for Compositional Zero-Shot Learning
  6. 2025Prompt-Based Continual Compositional Zero-Shot Learning
  7. 2022Do Vision-Language Pretrained Models Learn Composable Primitive Concepts?
  8. 2024Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation