# Zero-Shot Object Detection: Learning to Simultaneously Recognize and Localize Novel Concepts

arXiv: https://arxiv.org/abs/1803.06049 · HTML: https://mlsanity.com/paper/1803.06049/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2018.

## Built on by (most cited first, 30 of 48)

- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation](https://mlsanity.com/paper/2203.10421/) (2022)
- [Learning Open-World Object Proposals without Learning to Classify](https://mlsanity.com/paper/2108.06753/) (2021)
- [Open-World Entity Segmentation](https://mlsanity.com/paper/2107.14228/) (2021)
- [SATR: Zero-Shot Semantic Segmentation of 3D Shapes](https://mlsanity.com/paper/2304.04909/) (2023)
- [A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Future](https://mlsanity.com/paper/2307.09220/) (2023)
- [Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuning](https://mlsanity.com/paper/2208.08165/) (2022)
- [Opening up Open-World Tracking](https://mlsanity.com/paper/2104.11221/) (2021)
- [Better Call SAL: Towards Learning to Segment Anything in Lidar](https://mlsanity.com/paper/2403.13129/) (2024)
- [X-DETR: A Versatile Architecture for Instance-wise Vision-Language Tasks](https://mlsanity.com/paper/2204.05626/) (2022)
- [OvarNet: Towards Open-vocabulary Object Attribute Recognition](https://mlsanity.com/paper/2301.09506/) (2023)
- [SIGN: Spatial-information Incorporated Generative Network for Generalized Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2108.12517/) (2021)
- [Zero-Shot Instance Segmentation](https://mlsanity.com/paper/2104.06601/) (2021)
- [Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention](https://mlsanity.com/paper/2303.15274/) (2023)
- [Exemplar Free Class Agnostic Counting](https://mlsanity.com/paper/2205.14212/) (2022)
- [Zero-shot object goal visual navigation](https://mlsanity.com/paper/2206.07423/) (2022)
- [Few-Shot Object Detection: A Comprehensive Survey](https://mlsanity.com/paper/2112.11699/) (2021)
- [Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentation](https://mlsanity.com/paper/2301.00805/) (2023)
- [Semantics-Guided Contrastive Network for Zero-Shot Object detection](https://mlsanity.com/paper/2109.06062/) (2021)
- [CapDet: Unifying Dense Captioning and Open-World Detection Pretraining](https://mlsanity.com/paper/2303.02489/) (2023)
- [Semantic Diversity Learning for Zero-Shot Multi-label Classification](https://mlsanity.com/paper/2105.05926/) (2021)
- [Semantic Segmentation In-the-Wild Without Seeing Any Segmentation Examples](https://mlsanity.com/paper/2112.03185/) (2021)
- [MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesis](https://mlsanity.com/paper/2407.13675/) (2024)
- [Robust Region Feature Synthesizer for Zero-Shot Object Detection](https://mlsanity.com/paper/2201.00103/) (2022)
- [Object Detection with Multimodal Large Vision-Language Models: An In-depth Review](https://mlsanity.com/paper/2508.19294/) (2025)
- [Learning to Discover and Detect Objects](https://mlsanity.com/paper/2210.10774/) (2022)
- [Zero-Shot Aerial Object Detection with Visual Description Regularization](https://mlsanity.com/paper/2402.18233/) (2024)
- [Zero-shot Object Detection Through Vision-Language Embedding Alignment](https://mlsanity.com/paper/2109.12066/) (2021)
- [Fine-Grained Zero-Shot Object Detection](https://mlsanity.com/paper/2507.10358/) (2025)
- [SeeDS: Semantic Separable Diffusion Synthesizer for Zero-shot Food Detection](https://mlsanity.com/paper/2310.04689/) (2023)

## Abstract (arXiv)

Current Zero-Shot Learning (ZSL) approaches are restricted to recognition of a single dominant unseen object category in a test image. We hypothesize that this setting is ill-suited for real-world applications where unseen objects appear only as a part of a complex scene, warranting both the `recognition' and `localization' of an unseen category. To address this limitation, we introduce a new `Zero-Shot Detection' (ZSD) problem setting, which aims at simultaneously recognizing and locating object instances belonging to novel categories without any training examples. We also propose a new experimental protocol for ZSD based on the highly challenging ILSVRC dataset, adhering to practical issues, e.g., the rarity of unseen objects. To the best of our knowledge, this is the first end-to-end deep network for ZSD that jointly models the interplay between visual and semantic domain information. To overcome the noise in the automatically derived semantic descriptions, we utilize the concept of meta-classes to design an original loss function that achieves synergy between max-margin class separation and semantic space clustering. Furthermore, we present a baseline approach extended from recognition to detection setting. Our extensive experiments show significant performance boost over the baseline on the imperative yet difficult ZSD problem.