mlsanityData to 7 Oct 2026

SimLabel: Consistency-Guided OOD Detection with Pretrained Vision-Language Models

arXiv 2501.11485 ↗First posted 20 Jan 2025cs.CVcited by 5

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Detecting out-of-distribution (OOD) data is crucial in real-world machine learning applications, particularly in safety-critical domains. Existing methods often leverage language information from vision-language models (VLMs) to enhance OOD detection by improving confidence estimation through rich class-wise text information. However, when building OOD detection score upon on in-distribution (ID) text-image affinity, existing works either focus on each ID class or whole ID label sets, overlooking inherent ID classes' connection. We find that the semantic information across different ID classes is beneficial for effective OOD detection. We thus investigate the ability of image-text comprehension among different semantic-related ID labels in VLMs and propose a novel post-hoc strategy called SimLabel. SimLabel enhances the separability between ID and OOD samples by establishing a more robust image-class similarity metric that considers consistency over a set of similar class labels. Extensive experiments demonstrate the superior performance of SimLabel on various zero-shot OOD detection benchmarks. The proposed model is also extended to various VLM-backbones, demonstrating its good generalization ability. Our demonstration and implementation codes are available at: https://github.com/ShuZou-1/SimLabel.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 39 references that are in the map. First posted on arXiv 20 Jan 2025.

Builds on

29 of 39 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Negative Label Guided OOD Detection with Pretrained Vision-Language Models29 Mar

Other references · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  7. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  8. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  9. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  10. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  11. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  12. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
16 more
  1. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  2. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  3. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  4. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  5. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  6. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  7. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  8. 2021Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIPcited by 151
  9. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  10. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 84
  11. 2016Places: An Image Database for Deep Scene Understandingcited by 76
  12. 2022Improving Zero-shot Generalization and Robustness of Multi-modal Modelscited by 44
  13. 2021Unsupervised Out-of-Domain Detection via Pre-trained Transformerscited by 31
  14. 2021Enhancing the Generalization for Intent Classification and Out-of-Domain Detection in SLUcited by 22
  15. 2022Towards Textual Out-of-Domain Detection without In-Domain Labelscited by 14
  16. 2023Exploring Large Language Models for Multi-Modal Out-of-Distribution Detectioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Modelscited by 26
  2. 2025Unlocking Vision-Language Models for Video Anomaly Detection via Fine-Grained Promptingcited by 8
  3. 2025High-Entropy Tokens as Multimodal Failure Points in Vision-Language Modelscited by 1
  4. 2026Mind the Way You Select Negative Texts: Pursuing the Distance Consistency in OOD Detection with VLMscited by 1
  5. 2026Break the Brake, Not the Wheel: Untargeted Jailbreak via Entropy Maximizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Harnessing Large Language and Vision-Language Models for Robust Out-of-Distribution Detection
  2. 2024Conjugated Semantic Pool Improves OOD Detection with Pre-trained Vision-Language Models
  3. 2024CMOOD: Concept-based Multi-label OOD Detection
  4. 2025An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivity
  5. 2026Respecting Modality Gap in Post-hoc Out-of-distribution Detection with Pre-trained Vision-Language Models
  6. 2025Out-of-Distribution Detection with Positive and Negative Prompt Supervision Using Large Language Models
  7. 2025ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoning
  8. 2024LAPT: Label-driven Automated Prompt Tuning for OOD Detection with Vision-Language Models