mlsanityData to 14 Oct 2026

Anticipating Future Object Compositions without Forgetting

arXiv 2407.10723 ↗First posted 15 Jul 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Despite the significant advancements in computer vision models, their ability to generalize to novel object-attribute compositions remains limited. Existing methods for Compositional Zero-Shot Learning (CZSL) mainly focus on image classification. This paper aims to enhance CZSL in object detection without forgetting prior learned knowledge. We use Grounding DINO and incorporate Compositional Soft Prompting (CSP) into it and extend it with Compositional Anticipation. We achieve a 70.5% improvement over CSP on the harmonic mean (HM) between seen and unseen compositions on the CLEVR dataset. Furthermore, we introduce Contrastive Prompt Tuning to incrementally address model confusion between similar compositions. We demonstrate the effectiveness of this method and achieve an increase of 14.5% in HM across the pretrain, increment, and unseen sets. Collectively, these methods provide a framework for learning various compositions with limited data, as well as improving the performance of underperforming compositions when additional data becomes available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 31 references that are in the map. First posted on arXiv 15 Jul 2024.

Builds on

25 of 31 references.

References · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  6. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  7. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  8. 2021Grounded Language-Image Pre-trainingcited by 1,156
  9. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  10. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
  11. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  12. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 183
13 more
  1. 2021Open World Compositional Zero-Shot Learningcited by 101
  2. 2023Learning without Forgetting for Vision-Language Modelscited by 79
  3. 2022Learning to Compose Soft Prompts for Compositional Zero-Shot Learningcited by 67
  4. 2022Disentangling Visual Embeddings for Attributes and Objectscited by 55
  5. 2022Siamese Contrastive Embedding Network for Compositional Zero-Shot Learningcited by 55
  6. 2023OvarNet: Towards Open-vocabulary Object Attribute Recognitioncited by 43
  7. 2021Independent Prototype Propagation for Zero-Shot Compositionalitycited by 42
  8. 2022Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundariescited by 31
  9. 2023Learning Conditional Attributes for Compositional Zero-Shot Learningcited by 28
  10. 2023DRPT: Disentangled and Recurrent Prompt Tuning for Compositional Zero-Shot Learningcited by 17
  11. 2023How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detectioncited by 15
  12. 2023CAILA: Concept-Aware Intra-Layer Adapters for Compositional Zero-Shot Learningcited by 7
  13. 2022Reference-Limited Compositional Zero-Shot Learningcited by 3

Built on by

Not cited yet.