mlsanityData to 14 Oct 2026

Near OOD Detection for Vision-Language Prompt Learning with Contrastive Logit Score

arXiv 2405.16091 ↗First posted 25 May 2024cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Prompt learning has emerged as an efficient and effective method for fine-tuning vision-language models such as CLIP. While many studies have explored generalisation abilities of these models in few-shot classification tasks and a few studies have addressed far out-of-distribution (OOD) of the models, their potential for addressing near OOD detection remains underexplored. Existing methods either require training from scratch, need fine-tuning, or are not designed for vision-language prompt learning. To address this, we introduce the Contrastive Logit Score (CLS), a novel post-hoc, plug-and-play scoring function. CLS significantly improves near OOD detection of pre-trained vision-language prompt learning methods without modifying their model architectures or requiring retraining. Our method achieves up to an 11.67% improvement in AUROC for near OOD detection with minimal computational overhead. Extensive evaluations validate the effectiveness, efficiency, and generalisability of our approach. Our code is available at https://github.com/davidmcjung/near-OOD-prompt-learning.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 52 of its 73 references that are in the map. First posted on arXiv 25 May 2024.

Builds on

52 of 73 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Visual-Language Prompt Tuning with Knowledge-guided Context Optimization23 Mar
  2. 2022Prompt-aligned Gradient for Prompt Tuning30 May
  3. 2021Learning to Prompt for Vision-Language Models2 Sep

Other references · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  10. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  11. 2013Describing Textures in the Wildcited by 2,125
  12. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
37 more
  1. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  2. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  3. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  4. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  5. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,274
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  7. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  8. 2014Deep Visual-Semantic Alignments for Generating Image Descriptionscited by 968
  9. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
  10. 2021Generalized Out-of-Distribution Detection: A Surveycited by 802
  11. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 798
  12. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 600
  13. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  14. 2018Predictive Uncertainty Estimation via Prior Networkscited by 557
  15. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 531
  16. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  17. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 405
  18. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 339
  19. 2018Recent Advances in Open Set Recognition: A Surveycited by 310
  20. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  21. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  22. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 268
  23. 2016Image Captioning with Semantic Attentioncited by 260
  24. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 216
  25. 2021A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detectioncited by 207
  26. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  27. 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
  28. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  29. 2021Machine Learning with a Reject Option: A surveycited by 109
  30. 2023LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learningcited by 107
  31. 2015The Extreme Value Machinecited by 78
  32. 2021Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-grained Environmentscited by 66
  33. 2022Full-Spectrum Out-of-Distribution Detectioncited by 63
  34. 2020Classification with Rejection Based on Cost-sensitive Classificationcited by 55
  35. 2019On the Calibration of Multiclass Classification with Rejectioncited by 41
  36. 2020Towards Maximizing the Representation Gap between In-Domain & Out-of-Distribution Examplescited by 40
  37. 2016Image Captioning with Deep Bidirectional LSTMscited by 21

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  2. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 26

Similar, not cited

Close in wording, no citation either way

  1. 2023How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models?
  2. 2024LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models
  3. 2025FA: Forced Prompt Learning of Vision-Language Models for Out-of-Distribution Detection
  4. 2025Harnessing Large Language and Vision-Language Models for Robust Out-of-Distribution Detection
  5. 2024Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detection
  6. 2023Adapting Contrastive Language-Image Pretrained (CLIP) Models for Out-of-Distribution Detection
  7. 2024Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detection
  8. 2025Out-of-Distribution Detection with Positive and Negative Prompt Supervision Using Large Language Models