# Open-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning

arXiv: https://arxiv.org/abs/2211.11530 · HTML: https://mlsanity.com/paper/2211.11530/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [Open-Set Recognition: a Good Closed-Set Classifier is All You Need?](https://mlsanity.com/paper/2110.06207/) (2021)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Towards Open World Recognition](https://mlsanity.com/paper/1412.5687/) (2014)
- [FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding](https://mlsanity.com/paper/2103.05950/) (2021)
- [OpenGAN: Open-Set Recognition via Open Data Generation](https://mlsanity.com/paper/2104.02939/) (2021)
- [Contrastive Learning based Hybrid Networks for Long-Tailed Image Classification](https://mlsanity.com/paper/2103.14267/) (2021)
- [OW-DETR: Open-world Detection Transformer](https://mlsanity.com/paper/2112.01513/) (2021)
- [Learning Placeholders for Open-Set Recognition](https://mlsanity.com/paper/2103.15086/) (2021)
- [Dropout Sampling for Robust Object Detection in Open-Set Conditions](https://mlsanity.com/paper/1710.06677/) (2017)
- [Learning Open-World Object Proposals without Learning to Classify](https://mlsanity.com/paper/2108.06753/) (2021)
- [Conditional Gaussian Distribution Learning for Open Set Recognition](https://mlsanity.com/paper/2003.08823/) (2020)
- [Contrastive Embedding for Generalized Zero-Shot Learning](https://mlsanity.com/paper/2103.16173/) (2021)
- [Contrastive Learning for Label-Efficient Semantic Segmentation](https://mlsanity.com/paper/2012.06985/) (2020)
- [UC-OWOD: Unknown-Classified Open World Object Detection](https://mlsanity.com/paper/2207.11455/) (2022)
- [Expanding Low-Density Latent Regions for Open-Set Object Detection](https://mlsanity.com/paper/2203.14911/) (2022)
- [Uncertainty for Identifying Open-Set Errors in Visual Object Detection](https://mlsanity.com/paper/2104.01328/) (2021)

## Built on by (most cited first, 7 of 7)

- [Open-Set Recognition in the Age of Vision-Language Models](https://mlsanity.com/paper/2403.16528/) (2024)
- [Grasp, See, and Place: Efficient Unknown Object Rearrangement with Policy Structure Prior](https://mlsanity.com/paper/2402.15402/) (2024)
- [From Open-Vocabulary to Vocabulary-Free Semantic Segmentation](https://mlsanity.com/paper/2502.11891/) (2025)
- [A Light-Weight Framework for Open-Set Object Detection with Decoupled Feature Alignment in Joint Space](https://mlsanity.com/paper/2412.14680/) (2024)
- [Efficient Alignment of Unconditioned Action Prior for Language-conditioned Pick and Place in Clutter](https://mlsanity.com/paper/2503.09423/) (2025)
- [Open-Set Object Detection By Aligning Known Class Representations](https://mlsanity.com/paper/2412.20701/) (2024)
- [Model-Agnostic Open-Set Air-to-Air Visual Object Detection for Reliable UAV Perception](https://mlsanity.com/paper/2509.09297/) (2025)

## Abstract (arXiv)

Detecting both known and unknown objects is a fundamental skill for robot manipulation in unstructured environments. Open-set object detection (OSOD) is a promising direction to handle the problem consisting of two subtasks: objects and background separation, and open-set object classification. In this paper, we present Openset RCNN to address the challenging OSOD. To disambiguate unknown objects and background in the first subtask, we propose to use classification-free region proposal network (CF-RPN) which estimates the objectness score of each region purely using cues from object's location and shape preventing overfitting to the training categories. To identify unknown objects in the second subtask, we propose to represent them using the complementary region of known categories in a latent space which is accomplished by a prototype learning network (PLN). PLN performs instance-level contrastive learning to encode proposals to a latent space and builds a compact region centering with a prototype for each known category. Further, we note that the detection performance of unknown objects can not be unbiasedly evaluated on the situation that commonly used object detection datasets are not fully annotated. Thus, a new benchmark is introduced by reorganizing GraspNet-1billion, a robotic grasp pose detection dataset with complete annotation. Extensive experiments demonstrate the merits of our method. We finally show that our Openset RCNN can endow the robot with an open-set perception ability to support robotic rearrangement tasks in cluttered environments. More details can be found in https://sites.google.com/view/openset-rcnn/