mlsanityData to 7 Oct 2026

Open-Vocabulary Object Detectors: Robustness Challenges under Distribution Shifts

arXiv 2405.14874 ↗First posted 1 Apr 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

The challenge of Out-Of-Distribution (OOD) robustness remains a critical hurdle towards deploying deep vision models. Vision-Language Models (VLMs) have recently achieved groundbreaking results. VLM-based open-vocabulary object detection extends the capabilities of traditional object detection frameworks, enabling the recognition and classification of objects beyond predefined categories. Investigating OOD robustness in recent open-vocabulary object detection is essential to increase the trustworthiness of these models. This study presents a comprehensive robustness evaluation of the zero-shot capabilities of three recent open-vocabulary (OV) foundation object detection models: OWL-ViT, YOLO World, and Grounding DINO. Experiments carried out on the robustness benchmarks COCO-O, COCO-DC, and COCO-C encompassing distribution shifts due to information loss, corruption, adversarial attacks, and geometrical deformation, highlighting the challenges of the model's robustness to foster the research for achieving robustness. Project page: https://prakashchhipa.github.io/projects/ovod_robustness

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 44 of its 49 references that are in the map. First posted on arXiv 1 Apr 2024.

Builds on

44 of 49 references.

References · 44

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2023Segment Anythingcited by 8,158
  8. 2020End-to-End Object Detection with Transformerscited by 7,384
  9. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  12. 2015Fast R-CNNcited by 2,930
32 more
  1. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  2. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  3. 2019Natural Adversarial Examplescited by 1,401
  4. 2021Grounded Language-Image Pre-trainingcited by 1,147
  5. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  6. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 612
  7. 2024YOLO-World: Real-Time Open-Vocabulary Object Detectioncited by 498
  8. 2020Open-Vocabulary Object Detection Using Captionscited by 427
  9. 2023Scaling Open-Vocabulary Object Detectioncited by 335
  10. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  11. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  12. 2021Unsolved Problems in ML Safetycited by 291
  13. 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
  14. 2022F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Modelscited by 149
  15. 2022PromptDet: Towards Open-vocabulary Detection using Uncurated Imagescited by 133
  16. 2018Benchmarking Neural Network Robustness to Common Corruptions and Surface Variationscited by 99
  17. 2022Exploiting Unlabeled Data with Vision and Language Models for Object Detectioncited by 97
  18. 2023Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformerscited by 91
  19. 2023DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignmentcited by 90
  20. 2023Object-Aware Distillation Pyramid for Open-Vocabulary Object Detectioncited by 63
  21. 2023Multi-Modal Classifiers for Open-Vocabulary Object Detectioncited by 53
  22. 2022ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotationscited by 44
  23. 2023COCO-O: A Benchmark for Object Detectors under Natural Distribution Shiftscited by 34
  24. 2023Contrastive Feature Masking Open-Vocabulary Vision Transformercited by 31
  25. 2023ImageNet-E: Benchmarking Neural Network Robustness via Attribute Editingcited by 30
  26. 2022Open Vocabulary Object Detection with Proposal Mining and Prediction Equalizationcited by 23
  27. 2022Localized Vision-Language Matching for Open-vocabulary Object Detectioncited by 17
  28. 2023Three ways to improve feature alignment for open vocabulary detectioncited by 17
  29. 2023Open-Vocabulary Object Detection using Pseudo Caption Labelscited by 13
  30. 2023Open-Vocabulary Object Detection via Scene Graph Discoverycited by 7
  31. 2023Can Self-Supervised Representation Learning Methods Withstand Distribution Shifts and Corruptions?cited by 5
  32. 2024ObjectCompose: Evaluating Resilience of Vision-Based Models on Object-to-Background Compositional Changescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Object Detection with Multimodal Large Vision-Language Models: An In-depth Reviewcited by 14
  2. 2026Robust Onion: Peeling Open Vocab Object Detectors Under Noisecited by 1
  3. 2026Towards Domain-Generalized Open-Vocabulary Object Detection: A Progressive Domain-invariant Cross-modal Alignment Methodcited by 0
  4. 2026PISA: A Pseudo-Individual Source-Domain Feature Adaptation Framework for Test-Time Open-Vocabulary Object Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025An Empirical Analysis of VLM-based OOD Detection: Mechanisms, Advantages, and Sensitivity
  2. 2026Exploring Hierarchical Consistency and Unbiased Objectness for Open-Vocabulary Object Detection
  3. 2025ODOV: Benchmark the Open-Domain Open-Vocabulary Object Detection
  4. 2025Harnessing Large Language and Vision-Language Models for Robust Out-of-Distribution Detection
  5. 2024Generalized Out-of-Distribution Detection and Beyond in Vision Language Model Era: A Survey
  6. 2023How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection
  7. 2023What Makes Good Open-Vocabulary Detector: A Disassembling Perspective
  8. 2025VisTa: Visual-contextual and Text-augmented Zero-shot Object-level OOD Detection