# Open World DETR: Transformer based Open World Object Detection

arXiv: https://arxiv.org/abs/2212.02969 · HTML: https://mlsanity.com/paper/2212.02969/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 6 Dec 2022.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Towards Open World Recognition](https://mlsanity.com/paper/1412.5687/) (2014)
- [Incremental Learning of Object Detectors without Catastrophic Forgetting](https://mlsanity.com/paper/1708.06977/) (2017)
- [OW-DETR: Open-world Detection Transformer](https://mlsanity.com/paper/2112.01513/) (2021)
- [Faster ILOD: Incremental Learning for Object Detectors based on Faster RCNN](https://mlsanity.com/paper/2003.03901/) (2020)
- [Incremental Object Detection via Meta-Learning](https://mlsanity.com/paper/2003.08798/) (2020)
- [Bridging Non Co-occurrence with Unlabeled In-the-wild Data for Incremental Object Detection](https://mlsanity.com/paper/2110.15017/) (2021)

## Built on by (most cited first, 10 of 10)

- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [VLM-PL: Advanced Pseudo Labeling Approach for Class Incremental Object Detection via Vision-Language Model](https://mlsanity.com/paper/2403.05346/) (2024)
- [Open World Object Detection: A Survey](https://mlsanity.com/paper/2410.11301/) (2024)
- [Discovering Novel Actions from Open World Egocentric Videos with Object-Grounded Visual Commonsense Reasoning](https://mlsanity.com/paper/2305.16602/) (2023)
- [Unsupervised Recognition of Unknown Objects for Open-World Object Detection](https://mlsanity.com/paper/2308.16527/) (2023)
- [ProbRes: Probabilistic Jump Diffusion for Open-World Egocentric Activity Recognition](https://mlsanity.com/paper/2504.03948/) (2025)
- [Multi-modal Data based Semi-Supervised Learning for Vehicle Positioning](https://mlsanity.com/paper/2410.20680/) (2024)
- [Hallucinate, Ground, Repeat: A Framework for Generalized Visual Relationship Detection](https://mlsanity.com/paper/2506.05651/) (2025)
- [Parameterized Prompt for Incremental Object Detection](https://mlsanity.com/paper/2510.27316/) (2025)
- [Towards Open World Sound Event Detection](https://mlsanity.com/paper/2605.03934/) (2026)

## Abstract (arXiv)

Open world object detection aims at detecting objects that are absent in the object classes of the training data as unknown objects without explicit supervision. Furthermore, the exact classes of the unknown objects must be identified without catastrophic forgetting of the previous known classes when the corresponding annotations of unknown objects are given incrementally. In this paper, we propose a two-stage training approach named Open World DETR for open world object detection based on Deformable DETR. In the first stage, we pre-train a model on the current annotated data to detect objects from the current known classes, and concurrently train an additional binary classifier to classify predictions into foreground or background classes. This helps the model to build an unbiased feature representations that can facilitate the detection of unknown classes in subsequent process. In the second stage, we fine-tune the class-specific components of the model with a multi-view self-labeling strategy and a consistency constraint. Furthermore, we alleviate catastrophic forgetting when the annotations of the unknown classes becomes available incrementally by using knowledge distillation and exemplar replay. Experimental results on PASCAL VOC and MS-COCO show that our proposed method outperforms other state-of-the-art open world object detection methods by a large margin.