mlsanityData to 7 Oct 2026

Towards Open World Object Detection

arXiv 2103.02603 ↗First posted 3 Mar 2021cs.CV cs.AI cs.LGcited by 297

Not on the map. Why?

Abstract (arXiv)

Humans have a natural instinct to identify unknown object instances in their environments. The intrinsic curiosity about these unknown instances aids in learning about them, when the corresponding knowledge is eventually available. This motivates us to propose a novel computer vision problem called: `Open World Object Detection', where a model is tasked to: 1) identify objects that have not been introduced to it as `unknown', without explicit supervision to do so, and 2) incrementally learn these identified unknown categories without forgetting previously learned classes, when the corresponding labels are progressively received. We formulate the problem, introduce a strong evaluation protocol and provide a novel solution, which we call ORE: Open World Object Detector, based on contrastive clustering and energy based unknown identification. Our experimental evaluation and ablation studies analyze the efficacy of ORE in achieving Open World objectives. As an interesting by-product, we find that identifying and characterizing unknown instances helps to reduce confusion in an incremental object detection setting, where we achieve state-of-the-art performance, with no extra methodological effort. We hope that our work will attract further research into this newly identified, yet crucial research direction.

Key facts. Cited by 297 papers in the map (data through 7 Oct 2026). Builds on 41 of its 63 references that are in the map. First posted on arXiv 3 Mar 2021.

Builds on

41 of 63 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2017Mask R-CNNcited by 7,259
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  8. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  9. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  10. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  11. 2015Fast R-CNNcited by 2,930
  12. 2016Learning without Forgettingcited by 2,465
29 more
  1. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  2. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  3. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  4. 2016Progressive Neural Networkscited by 1,332
  5. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  6. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  7. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
  8. 2018Efficient Lifelong Learning with A-GEMcited by 1,044
  9. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 882
  10. 2017Focal Loss for Dense Object Detectioncited by 749
  11. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  12. 2018Overcoming catastrophic forgetting with hard attention to the taskcited by 672
  13. 2018End-to-End Incremental Learningcited by 596
  14. 2015Towards Open Set Deep Networkscited by 533
  15. 2020Frustratingly Simple Few-Shot Object Detectioncited by 280
  16. 2014Towards Open World Recognitioncited by 226
  17. 2017Incremental Learning of Object Detectors without Catastrophic Forgettingcited by 208
  18. 2017Generative OpenMax for Multi-Class Open Set Classificationcited by 157
  19. 2017Dropout Sampling for Robust Object Detection in Open-Set Conditionscited by 112
  20. 2018Generative Probabilistic Novelty Detection with Adversarial Autoencoderscited by 99
  21. 2020iTAML: An Incremental Task-Agnostic Meta-learning Approachcited by 90
  22. 2020Optimal Continual Learning has Perfect Memory and is NP-hardcited by 86
  23. 2020Faster ILOD: Incremental Learning for Object Detectors based on Faster RCNNcited by 61
  24. 2018Probabilistic Object Detection: Definition and Evaluationcited by 55
  25. 2018Evaluating Merging Strategies for Sampling-based Uncertainty Techniques in Object Detectioncited by 51
  26. 2020Meta-Consolidation for Continual Learningcited by 40
  27. 2018Open-world Learning and Application to Product Classificationcited by 39
  28. 2020RODEO: Replay for Online Object Detectioncited by 35
  29. 2017Memory Based Online Learning of Deep Representations from Video Streamscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 24; 2022: 65; 2023: 73; 2024: 55; 2025: 37; 2026: 43.

297 papers.

Sort:

By area:

  1. 2021Grounded Language-Image Pre-trainingcited by 1,147
  2. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  3. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  4. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  5. 2023Class-Incremental Learning: A Surveycited by 209
  6. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  7. 2022Bridging the Gap between Object and Image-level Representations for Open-Vocabulary Detectioncited by 145
  8. 2021OW-DETR: Open-world Detection Transformercited by 129
  9. 2021Learning Open-World Object Proposals without Learning to Classifycited by 109
  10. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  11. 2021Class-agnostic Object Detection with Multi-modal Transformercited by 85
  12. 2022UNICON: Combating Label Noise Through Uniform Selection and Contrastive Learningcited by 77
38 more
  1. 2021Open-World Entity Segmentationcited by 74
  2. 2022Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillationcited by 72
  3. 2021Open Vocabulary Object Detection with Pseudo Bounding-Box Labelscited by 71
  4. 2022Unknown-Aware Object Detection: Learning What You Don't Know from Videos in the Wildcited by 70
  5. 2023Continual Detection Transformer for Incremental Object Detectioncited by 59
  6. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 59
  7. 2022PROB: Probabilistic Objectness for Open World Object Detectioncited by 57
  8. 2023Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Reviewcited by 57
  9. 2021Recent Advances of Continual Learning in Computer Vision: An Overviewcited by 55
  10. 2022A Large-scale Multiple-objective Method for Black-box Attack against Object Detectioncited by 55
  11. 2023OVTrack: Open-Vocabulary Multiple Object Trackingcited by 53
  12. 2021Deep Metric Learning for Open World Semantic Segmentationcited by 51
  13. 2024A Hard-to-Beat Baseline for Training-free CLIP-based Adaptationcited by 51
  14. 2021Opening up Open-World Trackingcited by 50
  15. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
  16. 2022Towards Open-Set Object Detection and Discoverycited by 44
  17. 2022ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cuescited by 42
  18. 2022Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinitycited by 41
  19. 2022UC-OWOD: Unknown-Classified Open World Object Detectioncited by 40
  20. 2023Incremental 3D Semantic Scene Graph Prediction from RGB Sequencescited by 39
  21. 2022Expanding Low-Density Latent Regions for Open-Set Object Detectioncited by 38
  22. 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMscited by 38
  23. 2021Learning to Detect Every Thing in an Open Worldcited by 37
  24. 2022Energy-based Latent Aligner for Incremental Learningcited by 37
  25. 2023StepFormer: Self-supervised Step Discovery and Localization in Instructional Videoscited by 37
  26. 2023Learning Object State Changes in Videos: An Open-World Perspectivecited by 36
  27. 2022SAFE: Sensitivity-Aware Features for Out-of-Distribution Object Detectioncited by 33
  28. 2023Augmented Box Replay: Overcoming Foreground Shift for Incremental Object Detectioncited by 32
  29. 2022Revisiting Open World Object Detectioncited by 30
  30. 2021Open-set 3D Object Detectioncited by 29
  31. 2022Open-world Semantic Segmentation for LIDAR Point Cloudscited by 29
  32. 2023CAT: LoCalization and IdentificAtion Cascade Detection Transformer for Open-World Object Detectioncited by 28
  33. 2022RBC: Rectifying the Biased Context in Continual Semantic Segmentationcited by 27
  34. 2022Towards Realistic Semi-Supervised Learningcited by 27
  35. 2023Towards Continual Egocentric Activity Recognition: A Multi-modal Egocentric Activity Dataset for Continual Learningcited by 26
  36. 2023CapDet: Unifying Dense Captioning and Open-World Detection Pretrainingcited by 26
  37. 2024AIDE: An Automatic Data Engine for Object Detection in Autonomous Drivingcited by 26
  38. 2022Continual Object Detection: A review of definitions, strategies, and challengescited by 25

50 most cited of 297 All 297 (JSON)