mlsanityData to 7 Oct 2026

Negative Label Guided OOD Detection with Pretrained Vision-Language Models

arXiv 2403.20078 ↗First posted 29 Mar 2024cs.CV cs.LGcited by 66

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Out-of-distribution (OOD) detection aims at identifying samples from unknown classes, playing a crucial role in trustworthy models against errors on unexpected inputs. Extensive research has been dedicated to exploring OOD detection in the vision modality. Vision-language models (VLMs) can leverage both textual and visual information for various multi-modal applications, whereas few OOD detection methods take into account information from the text modality. In this paper, we propose a novel post hoc OOD detection method, called NegLabel, which takes a vast number of negative labels from extensive corpus databases. We design a novel scheme for the OOD score collaborated with negative labels. Theoretical analysis helps to understand the mechanism of negative labels. Extensive experiments demonstrate that our method NegLabel achieves state-of-the-art performance on various OOD detection benchmarks and generalizes well on multiple VLM architectures. Furthermore, our method NegLabel exhibits remarkable robustness against diverse domain shifts. The codes are available at https://github.com/tmlr-group/NegLabel.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 51 references that are in the map. First posted on arXiv 29 Mar 2024.

Builds on

35 of 51 references.

References · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2023Visual Instruction Tuningcited by 7,792
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  7. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  8. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  9. 2019Natural Adversarial Examplescited by 1,401
  10. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  11. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  12. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
23 more
  1. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  2. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  3. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  4. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  5. 2020CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instancescited by 441
  6. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  7. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
  8. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  9. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  10. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  11. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  12. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  13. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  14. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  15. 2021Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIPcited by 151
  16. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  17. 2023Non-Parametric Outlier Synthesiscited by 129
  18. 2021Semantically Coherent Out-of-Distribution Detectioncited by 107
  19. 2021Can multi-label classification networks know what they don't know?cited by 100
  20. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 84
  21. 2022Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wildcited by 70
  22. 2021On the Impact of Spurious Correlation for Out-of-distribution Detectioncited by 68
  23. 2021Energy-Based Open-World Uncertainty Modeling for Confidence Calibrationcited by 46

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 32; 2026: 19.

66 papers.

Sort:

By area:

  1. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Surveycited by 36
  2. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 24
  3. 2024LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Modelscited by 21
  4. 2024AdaNeg: Adaptive Negative Proxy Guided OOD Detection with Vision-Language Modelscited by 16
  5. 2025OODD: Test-time Out-of-Distribution Detection with Dynamic Dictionarycited by 16
  6. 2024Self-Calibrated Tuning of Vision-Language Models for Out-of-Distribution Detectioncited by 15
  7. 2024Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detectioncited by 14
  8. 2024CLIPScope: Enhancing Zero-Shot OOD Detection with Bayesian Scoringcited by 13
  9. 2024Conjugated Semantic Pool Improves OOD Detection with Pre-trained Vision-Language Modelscited by 13
  10. 2025ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoningcited by 10
  11. 2024SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximationcited by 9
  12. 2025Noisy Test-Time Adaptation in Vision-Language Modelscited by 8
38 more
  1. 2025Adaptive Multi-prompt Contrastive Network for Few-shot Out-of-distribution Detectioncited by 8
  2. 2025Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detectioncited by 6
  3. 2024From Open Vocabulary to Open World: Teaching Vision Language Models to Detect Novel Objectscited by 5
  4. 2025SimLabel: Consistency-Guided OOD Detection with Pretrained Vision-Language Modelscited by 5
  5. 2025FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQAcited by 5
  6. 2025Recent Advances in Out-of-Distribution Detection with CLIP-Like Models: A Surveycited by 5
  7. 2024VLMine: Long-Tail Data Mining with Vision Language Modelscited by 4
  8. 2024What If the Input is Expanded in OOD Detection?cited by 4
  9. 2025Outlier Synthesis via Hamiltonian Monte Carlo for Out-of-Distribution Detectioncited by 4
  10. 2025Revisiting Logit Distributions for Reliable Out-of-Distribution Detectioncited by 4
  11. 2025A Benchmark and Evaluation for Real-World Out-of-Distribution Detection Using Vision-Language Modelscited by 3
  12. 2025Mitigating the Modality Gap: Few-Shot Out-of-Distribution Detection with Multi-modal Prototypes and Image Bias Estimationcited by 3
  13. 2025Generalizable Prompt Learning of CLIP: A Brief Overviewcited by 3
  14. 2025To Trust Or Not To Trust Your Vision-Language Model's Predictioncited by 3
  15. 2025FA: Forced Prompt Learning of Vision-Language Models for Out-of-Distribution Detectioncited by 3
  16. 2025Synthesizing Near-Boundary OOD Samples for Out-of-Distribution Detectioncited by 3
  17. 2025Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracycited by 3
  18. 2026T-QPM: Enabling Temporal Out-Of-Distribution Detection and Domain Generalization for Vision-Language Models in Open-Worldcited by 3
  19. 2024Reflexive Guidance: Improving OoDD in Vision-Language Models via Self-Guided Image-Adaptive Concept Generationcited by 2
  20. 2025OpenEarthSensing: Large-Scale Fine-Grained Benchmark for Open-World Remote Sensingcited by 2
  21. 2025FOCoOp: Enhancing Out-of-Distribution Robustness in Federated Prompt Learning for Vision-Language Modelscited by 2
  22. 2025Spurious-Aware Prototype Refinement for Reliable Out-of-Distribution Detectioncited by 2
  23. 2025NegRefine: Refining Negative Label-Based Zero-Shot OOD Detectioncited by 2
  24. 2026TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-Language Modelscited by 2
  25. 2026Immuno-VLM: Immunizing Large Vision-Language Models via Generative Semantic Antibodies for Open-World Trustworthinesscited by 2
  26. 2025OT-DETECTOR: Delving into Optimal Transport for Zero-shot Out-of-Distribution Detectioncited by 1
  27. 2025An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivitycited by 1
  28. 2025Delta Energy: Optimizing Energy Change During Vision-Language Alignment Improves both OOD Detection and OOD Generalizationcited by 1
  29. 2025Leveraging Text-Driven Semantic Variation for Robust OOD Segmentationcited by 1
  30. 2026Mind the Way You Select Negative Texts: Pursuing the Distance Consistency in OOD Detection with VLMscited by 1
  31. 2026Debiased Negative Mining Improves Out-of-distribution Detection with Pre-trained Vision-Language Modelscited by 1
  32. 2026Dual Distribution Estimation for Zero-shot Noisy Test-Time Adaptation with VLMscited by 1
  33. 2024Textual Training for the Hassle-Free Removal of Unwanted Visual Data: Case Studies on OOD and Hateful Image Detectioncited by 0
  34. 2024Robust Noisy Correspondence Learning via Self-Drop and Dual-Weightcited by 0
  35. 2025Exploiting Vision Language Model for Training-Free 3D Point Cloud OOD Detection via Graph Score Propagationcited by 0
  36. 2025COOkeD: Ensemble-based OOD detection in the era of zero-shot CLIPcited by 0
  37. 2025OpenHAIV: A Framework Towards Practical Open-World Learningcited by 0
  38. 2025Retrieval-Augmented Prompt for OOD Detectioncited by 0

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Out-of-Distribution Detection with Positive and Negative Prompt Supervision Using Large Language Models
  2. 2025Harnessing Large Language and Vision-Language Models for Robust Out-of-Distribution Detection
  3. 2024CMOOD: Concept-based Multi-label OOD Detection
  4. 2026Activation Matters: Test-time Activated Negative Labels for OOD Detection with Vision-Language Models
  5. 2025Delving into Out-of-Distribution Detection with Medical Vision-Language Models
  6. 2023How Does Fine-Tuning Impact Out-of-Distribution Detection for Vision-Language Models?
  7. 2026Level, Sharpness, and Corpus: Why Zero-Shot OOD Detector Rankings Do Not Transfer
  8. 2024CRoFT: Robust Fine-Tuning with Concurrent Optimization for OOD Generalization and Open-Set OOD Detection