mlsanityData to 7 Oct 2026

Vision Also You Need: Navigating Out-of-Distribution Detection with Multimodal Large Language Model

arXiv 2601.14052 ↗First posted 20 Jan 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Out-of-Distribution (OOD) detection is a critical task that has garnered significant attention. The emergence of CLIP has spurred extensive research into zero-shot OOD detection, often employing a training-free approach. Current methods leverage expert knowledge from large language models (LLMs) to identify potential outliers. However, these approaches tend to over-rely on knowledge in the text space, neglecting the inherent challenges involved in detecting out-of-distribution samples in the image space. In this paper, we propose a novel pipeline, MM-OOD, which leverages the multimodal reasoning capabilities of MLLMs and their ability to conduct multi-round conversations for enhanced outlier detection. Our method is designed to improve performance in both near OOD and far OOD tasks. Specifically, (1) for near OOD tasks, we directly feed ID images and corresponding text prompts into MLLMs to identify potential outliers; and (2) for far OOD tasks, we introduce the sketch-generate-elaborate framework: first, we sketch outlier exposure using text prompts, then generate corresponding visual OOD samples, and finally elaborate by using multimodal prompts. Experiments demonstrate that our method achieves significant improvements on widely used multimodal datasets such as Food-101, while also validating its scalability on ImageNet-1K.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 53 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

36 of 53 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  7. 2013Describing Textures in the Wildcited by 2,110
  8. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  9. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  10. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  11. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  12. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
24 more
  1. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  2. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  3. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 215
  4. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  5. 2018Deep Learning for Single Image Super-Resolution: A Brief Reviewcited by 158
  6. 2021Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIPcited by 151
  7. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  8. 2023CLIPN for Zero-Shot OOD Detection: Teaching CLIP to Say Nocited by 130
  9. 2020ATOM: Robustifying Out-of-distribution Detection Using Outlier Miningcited by 116
  10. 2021MOOD: Multi-level Out-of-distribution Detectioncited by 94
  11. 2021Types of Out-of-Distribution Texts and How to Detect Themcited by 80
  12. 2021Contrastive Out-of-Distribution Detection for Pretrained Transformerscited by 70
  13. 2022Rethinking Reconstruction Autoencoder-Based Out-of-Distribution Detectioncited by 59
  14. 2019Out-of-Distribution Detection for Generalized Zero-Shot Action Recognitioncited by 54
  15. 2023ID-like Prompt Learning for Few-Shot Out-of-Distribution Detectioncited by 42
  16. 2023A Survey on Out-of-Distribution Detection in NLPcited by 31
  17. 2024Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detectioncited by 30
  18. 2022Watermarking for Out-of-distribution Detectioncited by 28
  19. 2024LOVA3: Learning to Visual Question Answering, Asking and Assessmentcited by 21
  20. 2023Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capabilitycited by 16
  21. 2024Long-Tailed Anomaly Detection with Learnable Class Namescited by 14
  22. 2024A noisy elephant in the room: Is your out-of-distribution detector robust to label noise?cited by 13
  23. 2024Gradient-Regularized Out-of-Distribution Detectioncited by 13
  24. 2023Genixer: Empowering Multimodal Large Language Models as a Powerful Data Generatorcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Recent Advances in Out-of-Distribution Detection with CLIP-Like Models: A Survey
  2. 2025Harnessing Large Language and Vision-Language Models for Robust Out-of-Distribution Detection
  3. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey
  4. 2024Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detection
  5. 2025Out-of-Distribution Detection with Positive and Negative Prompt Supervision Using Large Language Models
  6. 2023On the Powerfulness of Textual Outlier Exposure for Visual OoD Detection
  7. 2025Synthesizing Near-Boundary OOD Samples for Out-of-Distribution Detection
  8. 2024Zero-Shot Out-of-Distribution Detection with Outlier Label Exposure