mlsanityData to 14 Oct 2026

Immuno-VLM: Immunizing Large Vision-Language Models via Generative Semantic Antibodies for Open-World Trustworthiness

arXiv 2605.30745 ↗First posted 29 May 2026cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Large Vision-Language Models have achieved unprecedented success in zero-shot recognition by aligning visual features with broad semantic concepts. However, this semantic abstraction creates a critical vulnerability in open-world deployment: the ``Hubris of Semantics'', where models force-fit unknown anomalies into known categories with high confidence due to the lack of explicit negative knowledge. To address this textitOpen-World Trustworthiness Paradox, we propose Immuno-VLM, a bio-inspired framework that adapts the biological principle of Immunological Negative Selection to high-dimensional latent spaces. Departing from traditional Open-Set Recognition methods that rely on passive density estimation or inefficient pixel-space outlier generation, Immuno-VLM leverages the generative reasoning of Large Language Models to actively hallucinate ``Semantic Antibodies'', textual descriptions of near-distribution outliers (e.g., look-alikes, contextual anomalies) that effectively bound the decision space of known classes.Extensive experiments on ImageNet-1K and four challenging OOD benchmarks reveal that Immuno-VLM establishes a new state-of-the-art.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 85 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

47 of 85 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs27 May

Other references · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  5. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  6. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  7. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  8. 2019Natural Adversarial Examplescited by 1,408
  9. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  10. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  11. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  12. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
34 more
  1. 2020Generalized ODIN: Detecting Out-of-distribution Image without Learning from Out-of-distribution Datacited by 408
  2. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  3. 2020Why Normalizing Flows Fail to Detect Out-of-Distribution Datacited by 221
  4. 2022Delving into Out-of-Distribution Detection with Vision-Language Representationscited by 216
  5. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  6. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  7. 2017Generative OpenMax for Multi-Class Open Set Classificationcited by 157
  8. 2018Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifierscited by 126
  9. 2019Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancycited by 112
  10. 2024Negative Label Guided OOD Detection with Pretrained Vision-Language Modelscited by 66
  11. 2022Rethinking Reconstruction Autoencoder-Based Out-of-Distribution Detectioncited by 59
  12. 2022Multi-Modal Cross-Domain Alignment Network for Video Moment Retrievalcited by 43
  13. 2023You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videoscited by 41
  14. 2022Hierarchical Local-Global Transformer for Temporal Sentence Groundingcited by 38
  15. 2022Exploring Optical-Flow-Guided Motion and Detection-Based Appearance for Temporal Sentence Groundingcited by 35
  16. 2020Unbalanced Incomplete Multi-view Clustering via the Scheme of View Evolution: Weak Views are Meat; Strong Views do Eatcited by 30
  17. 2020ANIMC: A Soft Framework for Auto-weighted Noisy and Incomplete Multi-view Clusteringcited by 30
  18. 2018Hierarchical Novelty Detection for Visual Object Recognitioncited by 26
  19. 2023Hypotheses Tree Building for One-Shot Temporal Sentence Localizationcited by 25
  20. 2020Double Self-weighted Multi-view Clustering via Adaptive View Fusioncited by 23
  21. 2022Out-of-Distribution Detection with Semantic Mismatch under Maskingcited by 22
  22. 2024Your Data Is Not Perfect: Towards Cross-Domain Out-of-Distribution Detection in Class-Imbalanced Datacited by 21
  23. 2024Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Networkcited by 15
  24. 2025Taylor Series-Inspired Local Structure Fitting Network for Few-shot Point Cloud Semantic Segmentationcited by 14
  25. 2025To Align or Not to Align: Strategic Multimodal Representation Alignment for Optimal Performancecited by 14
  26. 2024Uncertainty-Guided Appearance-Motion Association Network for Out-of-Distribution Action Detectioncited by 12
  27. 2025Prototype-Driven Structure Synergy Network for Remote Sensing Images Segmentationcited by 11
  28. 2026Unveiling the Fragility of Vision-Language Models: Multi-Modal Adversarial Synergy via Texture-Constrained Perturbations and Cross-Modal Optimizationcited by 9
  29. 2025Adaptive Multi-prompt Contrastive Network for Few-shot Out-of-distribution Detectioncited by 8
  30. 2026Rethinking Weakly-supervised Video Temporal Grounding From a Game Perspectivecited by 8
  31. 2026Disentangling Adversarial Prompts: A Semantic-Graph Defense for Robust LLM Securitycited by 8
  32. 2026Rethinking Video-Language Model from the Language Input Perspectivecited by 6
  33. 2026Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Languagecited by 6
  34. 2026Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Languagecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Hierarchical Semantic-Augmented Navigation: Optimal Transport and Graph-Driven Reasoning for Vision-Language Navigationcited by 1
  2. 2026Turing Patterns for Multimedia: Reaction-Diffusion Multi-Modal Fusion for Language-Guided Video Moment Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Inductive Visual Logic for Few-Shot Out-Of-Distribution Adaptation in VLMs
  2. 2026Phantasia: Context-Adaptive Backdoors in Vision Language Models
  3. 2026TrustVLA: Mechanism-Guided Inference-Time Defense Against Vision-Language-Action Backdoors
  4. 2025HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models
  5. 2026ComplexityWorld: Benchmarking Vision-Language Models on Verifiable Visual Decision Making
  6. 2026UNBOX: Unveiling Black-box visual models with Natural-language
  7. 2024VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models
  8. 2024NPHardEval4V: Dynamic Evaluation of Large Vision-Language Models with Effects of Vision