# UFO2: A Unified Framework towards Omni-supervised Object Detection

arXiv: https://arxiv.org/abs/2010.10804 · HTML: https://mlsanity.com/paper/2010.10804/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2020.

## Built on by (most cited first, 26 of 26)

- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [Neural Volumetric Object Selection](https://mlsanity.com/paper/2205.14929/) (2022)
- [Points as Queries: Weakly Semi-supervised Object Detection by Points](https://mlsanity.com/paper/2104.07434/) (2021)
- [Exploring Simple 3D Multi-Object Tracking for Autonomous Driving](https://mlsanity.com/paper/2108.10312/) (2021)
- [Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization](https://mlsanity.com/paper/2108.05029/) (2021)
- [Self-supervised object detection from audio-visual correspondence](https://mlsanity.com/paper/2104.06401/) (2021)
- [PointOBB: Learning Oriented Object Detection via Single Point Supervision](https://mlsanity.com/paper/2311.14757/) (2023)
- [Active Learning Strategies for Weakly-supervised Object Detection](https://mlsanity.com/paper/2207.12112/) (2022)
- [Semantic-aware SAM for Point-Prompted Instance Segmentation](https://mlsanity.com/paper/2312.15895/) (2023)
- [Semi-supervised Active Learning for Video Action Detection](https://mlsanity.com/paper/2312.07169/) (2023)
- [BIMgent: Towards Autonomous Building Modeling via Computer-use Agents](https://mlsanity.com/paper/2506.07217/) (2025)
- [PointOBB-v3: Expanding Performance Boundaries of Single Point-Supervised Oriented Object Detection](https://mlsanity.com/paper/2501.13898/) (2025)
- [Learning Remote Sensing Object Detection with Single Point Supervision](https://mlsanity.com/paper/2305.14141/) (2023)
- [Complete Instances Mining for Weakly Supervised Instance Segmentation](https://mlsanity.com/paper/2402.07633/) (2024)
- [ORF-Net: Deep Omni-supervised Rib Fracture Detection from Chest CT Scans](https://mlsanity.com/paper/2207.01842/) (2022)
- [Neglected Free Lunch -- Learning Image Classifiers Using Annotation Byproducts](https://mlsanity.com/paper/2303.17595/) (2023)
- [OmViD: Omni-supervised active learning for video action detection](https://mlsanity.com/paper/2508.13983/) (2025)
- [DeLR: Active Learning for Detection with Decoupled Localization and Recognition Query](https://mlsanity.com/paper/2312.16931/) (2023)
- [Towards Universal Vision-language Omni-supervised Segmentation](https://mlsanity.com/paper/2303.06547/) (2023)
- [Deep Omni-supervised Learning for Rib Fracture Detection from Chest Radiology Images](https://mlsanity.com/paper/2306.13301/) (2023)
- [Towards Omni-supervised Referring Expression Segmentation](https://mlsanity.com/paper/2311.00397/) (2023)
- [Weakly-semi-supervised object detection in remotely sensed imagery](https://mlsanity.com/paper/2311.17449/) (2023)
- [Sparse Generation: Making Pseudo Labels Sparse for Point Weakly Supervised Object Detection on Low Data Volume](https://mlsanity.com/paper/2403.19306/) (2024)
- [ComplETR: Reducing the cost of annotations for object detection in dense scenes with vision transformers](https://mlsanity.com/paper/2209.05654/) (2022)
- [ABBSPO: Adaptive Bounding Box Scaling and Symmetric Prior based Orientation Prediction for Detecting Aerial Image Objects](https://mlsanity.com/paper/2512.10031/) (2025)
- [SAPNet++: Evolving Point-Prompted Instance Segmentation with Semantic and Spatial Awareness](https://mlsanity.com/paper/2602.21762/) (2026)

## Abstract (arXiv)

Existing work on object detection often relies on a single form of annotation: the model is trained using either accurate yet costly bounding boxes or cheaper but less expressive image-level tags. However, real-world annotations are often diverse in form, which challenges these existing works. In this paper, we present UFO2, a unified object detection framework that can handle different forms of supervision simultaneously. Specifically, UFO2 incorporates strong supervision (e.g., boxes), various forms of partial supervision (e.g., class tags, points, and scribbles), and unlabeled data. Through rigorous evaluations, we demonstrate that each form of label can be utilized to either train a model from scratch or to further improve a pre-trained model. We also use UFO2 to investigate budget-aware omni-supervised learning, i.e., various annotation policies are studied under a fixed annotation budget: we show that competitive performance needs no strong labels for all data. Finally, we demonstrate the generalization of UFO2, detecting more than 1,000 different objects without bounding box annotations.