# DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations

arXiv: https://arxiv.org/abs/2206.09541 · HTML: https://mlsanity.com/paper/2206.09541/

## Key facts

- Cited by 102 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 87 references that are in the map.
- First posted on arXiv 20 Jun 2022.

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [Exploring the Limits of Weakly Supervised Pretraining](https://mlsanity.com/paper/1805.00932/) (2018)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling](https://mlsanity.com/paper/2111.03930/) (2021)
- [Asymmetric Loss For Multi-Label Classification](https://mlsanity.com/paper/2009.14119/) (2020)
- [Multi-Label Image Recognition with Graph Convolutional Networks](https://mlsanity.com/paper/1904.03582/) (2019)
- [Prompting Visual-Language Models for Efficient Video Understanding](https://mlsanity.com/paper/2112.04478/) (2021)
- [Bilinear Attention Networks](https://mlsanity.com/paper/1805.07932/) (2018)
- [Prompt Distribution Learning](https://mlsanity.com/paper/2205.03340/) (2022)
- [CNN-RNN: A Unified Framework for Multi-label Image Classification](https://mlsanity.com/paper/1604.04573/) (2016)
- [Zero-Shot Object Detection](https://mlsanity.com/paper/1804.04340/) (2018)
- [CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models](https://mlsanity.com/paper/2109.11797/) (2021)
- [Label-Embedding for Image Classification](https://mlsanity.com/paper/1503.08677/) (2015)
- [Learning Visual Features from Large Weakly Supervised Data](https://mlsanity.com/paper/1511.02251/) (2015)
- [Zero-Shot Learning by Convex Combination of Semantic Embeddings](https://mlsanity.com/paper/1312.5650/) (2013)

## Built on by (most cited first, 30 of 102)

- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [AnomalyCLIP: Object-agnostic Prompt Learning for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2310.18961/) (2023)
- [Unified Vision and Language Prompt Learning](https://mlsanity.com/paper/2210.07225/) (2022)
- [DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models](https://mlsanity.com/paper/2310.16436/) (2023)
- [LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learning](https://mlsanity.com/paper/2306.01293/) (2023)
- [A Closer Look at the Explainability of Contrastive Language-Image Pre-training](https://mlsanity.com/paper/2304.05653/) (2023)
- [Vita-CLIP: Video and text adaptive CLIP via Multimodal Prompting](https://mlsanity.com/paper/2304.03307/) (2023)
- [Open-Set Image Tagging with Multi-Grained Text Supervision](https://mlsanity.com/paper/2310.15200/) (2023)
- [BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP](https://mlsanity.com/paper/2311.16194/) (2023)
- [Sentence-level Prompts Benefit Composed Image Retrieval](https://mlsanity.com/paper/2310.05473/) (2023)
- [Bayesian Prompt Learning for Image-Language Model Generalization](https://mlsanity.com/paper/2210.02390/) (2022)
- [Texts as Images in Prompt Tuning for Multi-Label Image Recognition](https://mlsanity.com/paper/2211.12739/) (2022)
- [Learning Transferable Negative Prompts for Out-of-Distribution Detection](https://mlsanity.com/paper/2404.03248/) (2024)
- [Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models](https://mlsanity.com/paper/2211.02219/) (2022)
- [CLIP in Medical Imaging: A Survey](https://mlsanity.com/paper/2312.07353/) (2023)
- [Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector](https://mlsanity.com/paper/2503.20188/) (2025)
- [CDUL: CLIP-Driven Unsupervised Learning for Multi-Label Image Classification](https://mlsanity.com/paper/2307.16634/) (2023)
- [GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detection](https://mlsanity.com/paper/2304.04521/) (2023)
- [Visual Tuning](https://mlsanity.com/paper/2305.06061/) (2023)
- [Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer](https://mlsanity.com/paper/2207.01887/) (2022)
- [Distilling Large Vision-Language Model with Out-of-Distribution Generalizability](https://mlsanity.com/paper/2307.03135/) (2023)
- [Enhancing CLIP with CLIP: Exploring Pseudolabeling for Limited-Label Prompt Tuning](https://mlsanity.com/paper/2306.01669/) (2023)
- [GalLoP: Learning Global and Local Prompts for Vision-Language Models](https://mlsanity.com/paper/2407.01400/) (2024)
- [PointAD: Comprehending 3D Anomalies from Points and Pixels for Zero-shot 3D Anomaly Detection](https://mlsanity.com/paper/2410.00320/) (2024)
- [DIME-FM: DIstilling Multimodal and Efficient Foundation Models](https://mlsanity.com/paper/2303.18232/) (2023)
- [Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels](https://mlsanity.com/paper/2303.13223/) (2023)
- [Defense-Prefix for Preventing Typographic Attacks on CLIP](https://mlsanity.com/paper/2304.04512/) (2023)
- [Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models](https://mlsanity.com/paper/2501.18592/) (2025)
- [DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations](https://mlsanity.com/paper/2308.01890/) (2023)
- [Why Is Prompt Tuning for Vision-Language Models Robust to Noisy Labels?](https://mlsanity.com/paper/2307.11978/) (2023)

## Abstract (arXiv)

Solving multi-label recognition (MLR) for images in the low-label regime is a challenging task with many real-world applications. Recent work learns an alignment between textual and visual spaces to compensate for insufficient image labels, but loses accuracy because of the limited amount of available MLR annotations. In this work, we utilize the strong alignment of textual and visual features pretrained with millions of auxiliary image-text pairs and propose Dual Context Optimization (DualCoOp) as a unified framework for partial-label MLR and zero-shot MLR. DualCoOp encodes positive and negative contexts with class names as part of the linguistic input (i.e. prompts). Since DualCoOp only introduces a very light learnable overhead upon the pretrained vision-language framework, it can quickly adapt to multi-label recognition tasks that have limited annotations and even unseen classes. Experiments on standard multi-label recognition benchmarks across two challenging low-label settings demonstrate the advantages of our approach over state-of-the-art methods.