Enhancing Novel Object Detection via Cooperative Foundational Models
Not on the map. Why?
Abstract (arXiv)
In this work, we address the challenging and emergent problem of novel object detection (NOD), focusing on the accurate detection of both known and novel object categories during inference. Traditional object detection algorithms are inherently closed-set, limiting their capability to handle NOD. We present a novel approach to transform existing closed-set detectors into open-set detectors. This transformation is achieved by leveraging the complementary strengths of pre-trained foundational models, specifically CLIP and SAM, through our cooperative mechanism. Furthermore, by integrating this mechanism with state-of-the-art open-set detectors such as GDINO, we establish new benchmarks in object detection performance. Our method achieves 17.42 mAP in novel object detection and 42.08 mAP for known objects on the challenging LVIS dataset. Adapting our approach to the COCO OVD split, we surpass the current state-of-the-art by a margin of 7.2 AP50 for novel classes. Our code is available at https://rohit901.github.io/coop-foundation-models/ .
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 38 references that are in the map. First posted on arXiv 19 Nov 2023.
Builds on
28 of 38 references.
References · 28
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2017Mask R-CNNcited by 7,277
- 2016Feature Pyramid Networks for Object Detectioncited by 5,036
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
- 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
- 2021Grounded Language-Image Pre-trainingcited by 1,156
- 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 906
- 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
- 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
16 more
- 2019Object Detection in 20 Years: A Surveycited by 531
- 2020Open-Vocabulary Object Detection Using Captionscited by 427
- 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
- 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
- 2022Open-Vocabulary DETR with Conditional Matchingcited by 220
- 2022Generalized Category Discoverycited by 211
- 2021Open-World Semi-Supervised Learningcited by 165
- 2021A Unified Objective for Novel Class Discoverycited by 162
- 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 146
- 2023Aligning Bag of Regions for Open-Vocabulary Object Detectioncited by 119
- 2022PromptCAL: Contrastive Affinity Learning via Auxiliary Prompts for Generalized Novel Category Discoverycited by 87
- 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
- 2021Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervisioncited by 39
- 2022OpenLDN: Learning to Discover Novel Classes for Open-World Semi-Supervised Learningcited by 37
- 2023Unknown Sniffer for Object Detection: Don't Turn a Blind Eye to Unknown Objectscited by 23
- 2022Learning to Discover and Detect Objectscited by 11
Built on by
Citing papers per year: 2026: 2.
2 papers.