mlsanityData to 14 Oct 2026

Occlusion Robustness of CLIP for Military Vehicle Classification

arXiv 2508.20760 ↗First posted 28 Aug 2025cs.CV cs.AIcited by 2

On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP

Abstract (arXiv)

Vision-language models (VLMs) like CLIP enable zero-shot classification by aligning images and text in a shared embedding space, offering advantages for defense applications with scarce labeled data. However, CLIP's robustness in challenging military environments, with partial occlusion and degraded signal-to-noise ratio (SNR), remains underexplored. We investigate CLIP variants' robustness to occlusion using a custom dataset of 18 military vehicle classes and evaluate using Normalized Area Under the Curve (NAUC) across occlusion percentages. Four key insights emerge: (1) Transformer-based CLIP models consistently outperform CNNs, (2) fine-grained, dispersed occlusions degrade performance more than larger contiguous occlusions, (3) despite improved accuracy, performance of linear-probed models sharply drops at around 35% occlusion, (4) by finetuning the model's backbone, this performance drop occurs at more than 60% occlusion. These results underscore the importance of occlusion-specific augmentations during training and the need for further exploration into patch-level sensitivity and architectural resilience for real-world deployment of CLIP.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 15 references that are in the map. First posted on arXiv 28 Aug 2025.

Builds on

10 of 15 references.

References · 10

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2013Fine-Grained Visual Classification of Aircraftcited by 1,726
  5. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  7. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  8. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  9. 2024Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Modelscited by 120
  10. 2023An Inverse Scaling Law for CLIP Trainingcited by 77

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Comparative Study of Out-of-the-Box Technology for Automatic Target Detection and Recognitioncited by 0
  2. 2026Beyond Benchmarks: Using VLMs to Reveal Systematic Classification Failures Under Real World Conditionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Toward a Holistic Evaluation of Robustness in CLIP Models
  2. 2023Improving CLIP Robustness with Knowledge Distillation and Self-Training
  3. 2025Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models
  4. 2024Calibrating Multi-modal Representations: A Pursuit of Group Robustness without Annotations
  5. 2025Navigating the Trade-off: A Synthesis of Defensive Strategies for Zero-Shot Adversarial Robustness in Vision-Language Models
  6. 2023Understanding the Vulnerability of CLIP to Image Compression
  7. 2025An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivity
  8. 2024A Hybrid Defense Strategy for Boosting Adversarial Robustness in Vision-Language Models