# Omni-DETR: Omni-Supervised Object Detection with Transformers

arXiv: https://arxiv.org/abs/2203.16089 · HTML: https://mlsanity.com/paper/2203.16089/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 53 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Unbiased Teacher for Semi-Supervised Object Detection](https://mlsanity.com/paper/2102.09480/) (2021)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [CrowdHuman: A Benchmark for Detecting Human in a Crowd](https://mlsanity.com/paper/1805.00123/) (2018)
- [A Simple Semi-Supervised Learning Framework for Object Detection](https://mlsanity.com/paper/2005.04757/) (2020)
- [Weakly Supervised Deep Detection Networks](https://mlsanity.com/paper/1511.02853/) (2015)
- [Humble Teachers Teach Better Students for Semi-Supervised Object Detection](https://mlsanity.com/paper/2106.10456/) (2021)
- [Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework](https://mlsanity.com/paper/2103.11402/) (2021)
- [PCL: Proposal Cluster Learning for Weakly Supervised Object Detection](https://mlsanity.com/paper/1807.03342/) (2018)
- [There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average](https://mlsanity.com/paper/1806.05594/) (2018)
- [Extreme clicking for efficient object annotation](https://mlsanity.com/paper/1708.02750/) (2017)
- [Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learning](https://mlsanity.com/paper/2101.08482/) (2021)
- [Points as Queries: Weakly Semi-supervised Object Detection by Points](https://mlsanity.com/paper/2104.07434/) (2021)

## Built on by (most cited first, 29 of 29)

- [GLIPv2: Unifying Localization and Vision-Language Understanding](https://mlsanity.com/paper/2206.05836/) (2022)
- [Learning Object-Language Alignments for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2211.14843/) (2022)
- [Semi-supervised Vision Transformers at Scale](https://mlsanity.com/paper/2208.05688/) (2022)
- [Semi-DETR: Semi-Supervised Object Detection with Detection Transformers](https://mlsanity.com/paper/2307.08095/) (2023)
- [Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve Aerial Visual Perception?](https://mlsanity.com/paper/2312.04548/) (2023)
- [Towards Unifying Anatomy Segmentation: Automated Generation of a Full-body CT Dataset via Knowledge Aggregation and Anatomical Guidelines](https://mlsanity.com/paper/2307.13375/) (2023)
- [Sparse Semi-DETR: Sparse Learnable Queries for Semi-Supervised Object Detection](https://mlsanity.com/paper/2404.01819/) (2024)
- [BSNet: Box-Supervised Simulation-assisted Mean Teacher for 3D Instance Segmentation](https://mlsanity.com/paper/2403.15019/) (2024)
- [ALWOD: Active Learning for Weakly-Supervised Object Detection](https://mlsanity.com/paper/2309.07914/) (2023)
- [How to Efficiently Annotate Images for Best-Performing Deep Learning Based Segmentation Models: An Empirical Study with Weak and Noisy Annotations and Segment Anything Model](https://mlsanity.com/paper/2312.10600/) (2023)
- [Semi-supervised Object Detection: A Survey on Recent Research and Progress](https://mlsanity.com/paper/2306.14106/) (2023)
- [Weakly Semi-Supervised Detection in Lung Ultrasound Videos](https://mlsanity.com/paper/2308.04463/) (2023)
- [What is Point Supervision Worth in Video Instance Segmentation?](https://mlsanity.com/paper/2404.01990/) (2024)
- [Adaptive Self-Training for Object Detection](https://mlsanity.com/paper/2212.05911/) (2022)
- [Improving Human-Object Interaction Detection via Virtual Image Learning](https://mlsanity.com/paper/2308.02606/) (2023)
- [End-to-End Semi-Supervised approach with Modulated Object Queries for Table Detection in Documents](https://mlsanity.com/paper/2405.04971/) (2024)
- [Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformer](https://mlsanity.com/paper/2407.08460/) (2024)
- [BOOTPLACE: Bootstrapped Object Placement with Detection Transformers](https://mlsanity.com/paper/2503.21991/) (2025)
- [OmViD: Omni-supervised active learning for video action detection](https://mlsanity.com/paper/2508.13983/) (2025)
- [DeLR: Active Learning for Detection with Decoupled Localization and Recognition Query](https://mlsanity.com/paper/2312.16931/) (2023)
- [Mixed-Query Transformer: A Unified Image Segmentation Architecture](https://mlsanity.com/paper/2404.04469/) (2024)
- [1st Place Solutions for the UVO Challenge 2022](https://mlsanity.com/paper/2210.09629/) (2022)
- [Towards Universal Vision-language Omni-supervised Segmentation](https://mlsanity.com/paper/2303.06547/) (2023)
- [Deep Omni-supervised Learning for Rib Fracture Detection from Chest Radiology Images](https://mlsanity.com/paper/2306.13301/) (2023)
- [Towards Omni-supervised Referring Expression Segmentation](https://mlsanity.com/paper/2311.00397/) (2023)
- [VFM-Guided Semi-Supervised Detection Transformer under Source-Free Constraints for Remote Sensing Object Detection](https://mlsanity.com/paper/2508.11167/) (2025)
- [Rethinking Scale Imbalance in Semi-supervised Object Detection for Aerial Images](https://mlsanity.com/paper/2310.14718/) (2023)
- [Seeking Flat Minima with Mean Teacher on Semi- and Weakly-Supervised Domain Generalization for Object Detection](https://mlsanity.com/paper/2310.19351/) (2023)
- [Bridging Supervision Gaps: A Unified Framework for Remote Sensing Change Detection](https://mlsanity.com/paper/2601.17747/) (2026)

## Abstract (arXiv)

We consider the problem of omni-supervised object detection, which can use unlabeled, fully labeled and weakly labeled annotations, such as image tags, counts, points, etc., for object detection. This is enabled by a unified architecture, Omni-DETR, based on the recent progress on student-teacher framework and end-to-end transformer based object detection. Under this unified architecture, different types of weak labels can be leveraged to generate accurate pseudo labels, by a bipartite matching based filtering mechanism, for the model to learn. In the experiments, Omni-DETR has achieved state-of-the-art results on multiple datasets and settings. And we have found that weak annotations can help to improve detection performance and a mixture of them can achieve a better trade-off between annotation cost and accuracy than the standard complete annotation. These findings could encourage larger object detection datasets with mixture annotations. The code is available at https://github.com/amazon-research/omni-detr.