# DeCoOp: Robust Prompt Tuning with Out-of-Distribution Detection

arXiv: https://arxiv.org/abs/2406.00345 · HTML: https://mlsanity.com/paper/2406.00345/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Prompt learning and adaptation of CLIP](https://mlsanity.com/mllm/prompt-learning-and-adaptation-of-clip/)

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models](https://mlsanity.com/paper/2209.07511/) (2022)
- [Online Continual Learning in Image Classification: An Empirical Survey](https://mlsanity.com/paper/2101.10423/) (2021)
- [Prompt Distribution Learning](https://mlsanity.com/paper/2205.03340/) (2022)
- [Patching open-vocabulary models by interpolating weights](https://mlsanity.com/paper/2208.05592/) (2022)
- [CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say No](https://mlsanity.com/paper/2308.12213/) (2023)
- [Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy](https://mlsanity.com/paper/1908.04951/) (2019)
- [LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learning](https://mlsanity.com/paper/2306.01293/) (2023)
- [Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-grained Environments](https://mlsanity.com/paper/2106.03917/) (2021)
- [Adaptivity and Non-stationarity: Problem-dependent Dynamic Regret for Online Convex Optimization](https://mlsanity.com/paper/2112.14368/) (2021)
- [Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurts](https://mlsanity.com/paper/2309.10019/) (2023)
- [Improving Zero-Shot Generalization for CLIP with Synthesized Prompts](https://mlsanity.com/paper/2307.07397/) (2023)
- [Black Box Few-Shot Adaptation for Vision-Language models](https://mlsanity.com/paper/2304.01752/) (2023)
- [How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models?](https://mlsanity.com/paper/2306.06048/) (2023)
- [CLIP-driven Outliers Synthesis for few-shot OOD detection](https://mlsanity.com/paper/2404.00323/) (2024)
- [LAMDA-SSL: Semi-Supervised Learning in Python](https://mlsanity.com/paper/2208.04610/) (2022)
- [Weak Distribution Detectors Lead to Stronger Generalizability of Vision-Language Prompt Tuning](https://mlsanity.com/paper/2404.00603/) (2024)

## Built on by (most cited first, 14 of 14)

- [Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey](https://mlsanity.com/paper/2407.21794/) (2024)
- [OpenworldAUC: Towards Unified Evaluation and Optimization for Open-world Prompt Tuning](https://mlsanity.com/paper/2505.05180/) (2025)
- [Robust Semi-Supervised Learning in Open Environments](https://mlsanity.com/paper/2412.18256/) (2024)
- [Generalizing vision-language models to novel domains: A comprehensive survey](https://mlsanity.com/paper/2506.18504/) (2025)
- [VCSearch: Bridging the Gap Between Well-Defined and Ill-Defined Problems in Mathematical Reasoning](https://mlsanity.com/paper/2406.05055/) (2024)
- [LIFT+: Lightweight Fine-Tuning for Long-Tail Learning](https://mlsanity.com/paper/2504.13282/) (2025)
- [BMIP: Bi-directional Modality Interaction Prompt Learning for VLM](https://mlsanity.com/paper/2501.07769/) (2025)
- [NeSy-Route: A Neuro-Symbolic Benchmark for Constrained Route Planning in Remote Sensing](https://mlsanity.com/paper/2603.16307/) (2026)
- [CoCoA-Mix: Confusion-and-Confidence-Aware Mixture Model for Context Optimization](https://mlsanity.com/paper/2506.07484/) (2025)
- [Delta Energy: Optimizing Energy Change During Vision-Language Alignment Improves both OOD Detection and OOD Generalization](https://mlsanity.com/paper/2510.11296/) (2025)
- [Move-Then-Operate: Behavioral Phasing for Human-Like Robotic Manipulation](https://mlsanity.com/paper/2604.23620/) (2026)
- [CGI: Identifying Conditional Generative Models with Example Images](https://mlsanity.com/paper/2501.13991/) (2025)
- [Boosting Open Set Recognition Performance through Modulated Representation Learning](https://mlsanity.com/paper/2505.18137/) (2025)
- [Bi-CoG: Bi-Consistency-Guided Self-Training for Vision-Language Models](https://mlsanity.com/paper/2510.20477/) (2025)

## Abstract (arXiv)

Vision-language models (VLMs), such as CLIP, have demonstrated impressive zero-shot capabilities for various downstream tasks. Their performance can be further enhanced through few-shot prompt tuning methods. However, current studies evaluate the performance of learned prompts separately on base and new classes. This evaluation lacks practicality for real-world applications since downstream tasks cannot determine whether the data belongs to base or new classes in advance. In this paper, we explore a problem setting called Open-world Prompt Tuning (OPT), which involves tuning prompts on base classes and evaluating on a combination of base and new classes. By introducing Decomposed Prompt Tuning framework (DePT), we theoretically demonstrate that OPT can be solved by incorporating out-of-distribution detection into prompt tuning, thereby enhancing the base-to-new discriminability. Based on DePT, we present a novel prompt tuning approach, namely, Decomposed Context Optimization (DeCoOp), which introduces new-class detectors and sub-classifiers to further enhance the base-class and new-class discriminability. Experimental results on 11 benchmark datasets validate the effectiveness of DePT and demonstrate that DeCoOp outperforms current state-of-the-art methods, providing a significant 2% average accuracy improvement.