# USD: Unknown Sensitive Detector Empowered by Decoupled Objectness and Segment Anything Model

arXiv: https://arxiv.org/abs/2306.02275 · HTML: https://mlsanity.com/paper/2306.02275/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 44 references that are in the map.
- First posted on arXiv 4 Jun 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [Segment Everything Everywhere All at Once](https://mlsanity.com/paper/2304.06718/) (2023)
- [VOS: Learning What You Don't Know by Virtual Outlier Synthesis](https://mlsanity.com/paper/2202.01197/) (2022)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Class-Incremental Learning: A Survey](https://mlsanity.com/paper/2302.03648/) (2023)
- [Incremental Learning of Object Detectors without Catastrophic Forgetting](https://mlsanity.com/paper/1708.06977/) (2017)
- [F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models](https://mlsanity.com/paper/2209.15639/) (2022)
- [Probabilistic two-stage detection](https://mlsanity.com/paper/2103.07461/) (2021)
- [OW-DETR: Open-world Detection Transformer](https://mlsanity.com/paper/2112.01513/) (2021)
- [Learning Open-World Object Proposals without Learning to Classify](https://mlsanity.com/paper/2108.06753/) (2021)
- [CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving](https://mlsanity.com/paper/2203.07724/) (2022)
- [Exploiting Unlabeled Data with Vision and Language Models for Object Detection](https://mlsanity.com/paper/2207.08954/) (2022)
- [Class-agnostic Object Detection with Multi-modal Transformer](https://mlsanity.com/paper/2111.11430/) (2021)
- [PROB: Probabilistic Objectness for Open World Object Detection](https://mlsanity.com/paper/2212.01424/) (2022)
- [Towards Open-Set Object Detection and Discovery](https://mlsanity.com/paper/2204.05604/) (2022)
- [UC-OWOD: Unknown-Classified Open World Object Detection](https://mlsanity.com/paper/2207.11455/) (2022)
- [Expanding Low-Density Latent Regions for Open-Set Object Detection](https://mlsanity.com/paper/2203.14911/) (2022)
- [Learning to Detect Every Thing in an Open World](https://mlsanity.com/paper/2112.01698/) (2021)
- [Multi-View Correlation Distillation for Incremental Object Detection](https://mlsanity.com/paper/2107.01787/) (2021)
- [Revisiting Open World Object Detection](https://mlsanity.com/paper/2201.00471/) (2022)
- [CAT: LoCalization and IdentificAtion Cascade Detection Transformer for Open-World Object Detection](https://mlsanity.com/paper/2301.01970/) (2023)

## Built on by (most cited first, 4 of 4)

- [Industrial Segment Anything -- a Case Study in Aircraft Manufacturing, Intralogistics, Maintenance, Repair, and Overhaul](https://mlsanity.com/paper/2307.12674/) (2023)
- [O1O: Grouping of Known Classes to Identify Unknown Objects as Odd-One-Out](https://mlsanity.com/paper/2410.07514/) (2024)
- [OW-Rep: Open World Object Detection with Instance Representation Learning](https://mlsanity.com/paper/2409.16073/) (2024)
- [Decoupled PROB: Decoupled Query Initialization Tasks and Objectness-Class Learning for Open World Object Detection](https://mlsanity.com/paper/2507.13085/) (2025)

## Abstract (arXiv)

Open World Object Detection (OWOD) is a novel and challenging computer vision task that enables object detection with the ability to detect unknown objects. Existing methods typically estimate the object likelihood with an additional objectness branch, but ignore the conflict in learning objectness and classification boundaries, which oppose each other on the semantic manifold and training objective. To address this issue, we propose a simple yet effective learning strategy, namely Decoupled Objectness Learning (DOL), which divides the learning of these two boundaries into suitable decoder layers. Moreover, detecting unknown objects comprehensively requires a large amount of annotations, but labeling all unknown objects is both difficult and expensive. Therefore, we propose to take advantage of the recent Large Vision Model (LVM), specifically the Segment Anything Model (SAM), to enhance the detection of unknown objects. Nevertheless, the output results of SAM contain noise, including backgrounds and fragments, so we introduce an Auxiliary Supervision Framework (ASF) that uses a pseudo-labeling and a soft-weighting strategies to alleviate the negative impact of noise. Extensive experiments on popular benchmarks, including Pascal VOC and MS COCO, demonstrate the effectiveness of our approach. Our proposed Unknown Sensitive Detector (USD) outperforms the recent state-of-the-art methods in terms of Unknown Recall, achieving significant improvements of 14.3%, 15.5%, and 8.9% on the M-OWODB, and 27.1%, 29.1%, and 25.1% on the S-OWODB.