# Random Boxes Are Open-world Object Detectors

arXiv: https://arxiv.org/abs/2307.08249 · HTML: https://mlsanity.com/paper/2307.08249/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 17 Jul 2023.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [DiffusionDet: Diffusion Model for Object Detection](https://mlsanity.com/paper/2211.09788/) (2022)
- [Towards Open World Object Detection](https://mlsanity.com/paper/2103.02603/) (2021)
- [Frustratingly Simple Few-Shot Object Detection](https://mlsanity.com/paper/2003.06957/) (2020)
- [Towards Open World Recognition](https://mlsanity.com/paper/1412.5687/) (2014)
- [OW-DETR: Open-world Detection Transformer](https://mlsanity.com/paper/2112.01513/) (2021)
- [Dropout Sampling for Robust Object Detection in Open-Set Conditions](https://mlsanity.com/paper/1710.06677/) (2017)
- [Learning Open-World Object Proposals without Learning to Classify](https://mlsanity.com/paper/2108.06753/) (2021)
- [iTAML: An Incremental Task-Agnostic Meta-learning Approach](https://mlsanity.com/paper/2003.11652/) (2020)
- [AugMax: Adversarial Composition of Random Augmentations for Robust Training](https://mlsanity.com/paper/2110.13771/) (2021)
- [Optimal Continual Learning has Perfect Memory and is NP-hard](https://mlsanity.com/paper/2006.05188/) (2020)
- [LSTD: A Low-Shot Transfer Detector for Object Detection](https://mlsanity.com/paper/1803.01529/) (2018)
- [Randomized Exploration in Generalized Linear Bandits](https://mlsanity.com/paper/1906.08947/) (2019)
- [PROB: Probabilistic Objectness for Open World Object Detection](https://mlsanity.com/paper/2212.01424/) (2022)
- [Evaluating Merging Strategies for Sampling-based Uncertainty Techniques in Object Detection](https://mlsanity.com/paper/1809.06006/) (2018)

## Built on by (most cited first, 24 of 24)

- [Does FLUX Already Know How to Perform Physically Plausible Image Composition?](https://mlsanity.com/paper/2509.21278/) (2025)
- [DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editing](https://mlsanity.com/paper/2510.02253/) (2025)
- [Open World Object Detection: A Survey](https://mlsanity.com/paper/2410.11301/) (2024)
- [Open World Object Detection in the Era of Foundation Models](https://mlsanity.com/paper/2312.05745/) (2023)
- [YOLO-UniOW: Efficient Universal Open-World Object Detection](https://mlsanity.com/paper/2412.20645/) (2024)
- [From Open Vocabulary to Open World: Teaching Vision Language Models to Detect Novel Objects](https://mlsanity.com/paper/2411.18207/) (2024)
- [SKDF: A Simple Knowledge Distillation Framework for Distilling Open-Vocabulary Knowledge to Open-world Object Detector](https://mlsanity.com/paper/2312.08653/) (2023)
- [Looking Beyond the Known: Towards a Data Discovery Guided Open-World Object Detection](https://mlsanity.com/paper/2510.00303/) (2025)
- [Boosting Few-Shot Open-Set Object Detection via Prompt Learning and Robust Decision Boundary](https://mlsanity.com/paper/2406.18443/) (2024)
- [Diffusion-Based Particle-DETR for BEV Perception](https://mlsanity.com/paper/2312.11578/) (2023)
- [UADet: A Remarkably Simple Yet Effective Uncertainty-Aware Open-Set Object Detection Framework](https://mlsanity.com/paper/2412.09229/) (2024)
- [Detecting Unknown Objects via Energy-based Separation for Open World Object Detection](https://mlsanity.com/paper/2603.29954/) (2026)
- [O1O: Grouping of Known Classes to Identify Unknown Objects as Odd-One-Out](https://mlsanity.com/paper/2410.07514/) (2024)
- [OW-CLIP: Data-Efficient Visual Supervision for Open-World Object Detection via Human-AI Collaboration](https://mlsanity.com/paper/2507.19870/) (2025)
- [VK-Det: Visual Knowledge Guided Prototype Learning for Open-Vocabulary Aerial Object Detection](https://mlsanity.com/paper/2511.18075/) (2025)
- [Knowing the Unknown: Interpretable Open-World Object Detection via Concept Decomposition Model](https://mlsanity.com/paper/2602.20616/) (2026)
- [EW-DETR: Evolving World Object Detection via Incremental Low-Rank DEtection TRansformer](https://mlsanity.com/paper/2602.20985/) (2026)
- [REAL-OW: Rehearsal-free Open World Object Detection with Low-Rank Adaptation and Dual-Stage Objectness Modeling](https://mlsanity.com/paper/2607.03004/) (2026)
- [OW-Rep: Open World Object Detection with Instance Representation Learning](https://mlsanity.com/paper/2409.16073/) (2024)
- [On the Inherent Robustness of One-Stage Object Detection against Out-of-Distribution Data](https://mlsanity.com/paper/2411.04586/) (2024)
- [Beyond Flat Unknown Labels in Open-World Object Detection](https://mlsanity.com/paper/2510.09173/) (2025)
- [DualMem: Bypassing the Objectness Bottleneck for Calibrated Unknown-Stream Filtering in Open-World Object Detection](https://mlsanity.com/paper/2605.23634/) (2026)
- [Towards Sparsely Annotated Open-World Object Detection](https://mlsanity.com/paper/2608.12714/) (2026)
- [Hyperbolic Geometry for Open-World Object Detection in Remote Sensing Imagery](https://mlsanity.com/paper/2609.09626/) (2026)

## Abstract (arXiv)

We show that classifiers trained with random region proposals achieve state-of-the-art Open-world Object Detection (OWOD): they can not only maintain the accuracy of the known objects (w/ training labels), but also considerably improve the recall of unknown ones (w/o training labels). Specifically, we propose RandBox, a Fast R-CNN based architecture trained on random proposals at each training iteration, surpassing existing Faster R-CNN and Transformer based OWOD. Its effectiveness stems from the following two benefits introduced by randomness. First, as the randomization is independent of the distribution of the limited known objects, the random proposals become the instrumental variable that prevents the training from being confounded by the known objects. Second, the unbiased training encourages more proposal explorations by using our proposed matching score that does not penalize the random proposals whose prediction scores do not match the known objects. On two benchmarks: Pascal-VOC/MS-COCO and LVIS, RandBox significantly outperforms the previous state-of-the-art in all metrics. We also detail the ablations on randomization and loss designs. Codes are available at https://github.com/scuwyh2000/RandBox.