# Points as Queries: Weakly Semi-supervised Object Detection by Points

arXiv: https://arxiv.org/abs/2104.07434 · HTML: https://mlsanity.com/paper/2104.07434/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [A Simple Semi-Supervised Learning Framework for Object Detection](https://mlsanity.com/paper/2005.04757/) (2020)
- [Weakly Supervised Deep Detection Networks](https://mlsanity.com/paper/1511.02853/) (2015)
- [TIDE: A General Toolbox for Identifying Object Detection Errors](https://mlsanity.com/paper/2008.08115/) (2020)
- [Proposal Learning for Semi-Supervised Object Detection](https://mlsanity.com/paper/2001.05086/) (2020)
- [Weakly Supervised Scene Parsing with Point-based Distance Metric Learning](https://mlsanity.com/paper/1811.02233/) (2018)
- [Towards Human-Machine Cooperation: Self-supervised Sample Mining for Object Detection](https://mlsanity.com/paper/1803.09867/) (2018)
- [Deep Self-Taught Learning for Weakly Supervised Object Localization](https://mlsanity.com/paper/1704.05188/) (2017)
- [UFO2: A Unified Framework towards Omni-supervised Object Detection](https://mlsanity.com/paper/2010.10804/) (2020)
- [Soft Proposal Networks for Weakly Supervised Object Localization](https://mlsanity.com/paper/1709.01829/) (2017)
- [Weakly Supervised Object Localization Using Things and Stuff Transfer](https://mlsanity.com/paper/1703.08000/) (2017)
- [Weakly- and Semi-Supervised Object Detection with Expectation-Maximization Algorithm](https://mlsanity.com/paper/1702.08740/) (2017)

## Built on by (most cited first, 30 of 60)

- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [Few-shot Object Counting and Detection](https://mlsanity.com/paper/2207.10988/) (2022)
- [Mapping Degeneration Meets Label Evolution: Learning Infrared Small Target Detection with Single Point Supervision](https://mlsanity.com/paper/2304.01484/) (2023)
- [Instance-Conditional Knowledge Distillation for Object Detection](https://mlsanity.com/paper/2110.12724/) (2021)
- [Omni-DETR: Omni-Supervised Object Detection with Transformers](https://mlsanity.com/paper/2203.16089/) (2022)
- [PointOBB: Learning Oriented Object Detection via Single Point Supervision](https://mlsanity.com/paper/2311.14757/) (2023)
- [Group R-CNN for Weakly Semi-supervised Object Detection with Points](https://mlsanity.com/paper/2205.05920/) (2022)
- [Pointly-Supervised Panoptic Segmentation](https://mlsanity.com/paper/2210.13950/) (2022)
- [The Devil is in the Points: Weakly Semi-Supervised Instance Segmentation via Point-Guided Mask Representation](https://mlsanity.com/paper/2303.15062/) (2023)
- [Point2RBox: Combine Knowledge from Synthetic Visual Patterns for End-to-end Oriented Object Detection with Single Point Supervision](https://mlsanity.com/paper/2311.14758/) (2023)
- [Monte Carlo Linear Clustering with Single-Point Supervision is Enough for Infrared Small Target Detection](https://mlsanity.com/paper/2304.04442/) (2023)
- [Active Learning Strategies for Weakly-supervised Object Detection](https://mlsanity.com/paper/2207.12112/) (2022)
- [Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learning](https://mlsanity.com/paper/2303.17842/) (2023)
- [Active Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2207.11493/) (2022)
- [Interactive Multi-Class Tiny-Object Detection](https://mlsanity.com/paper/2203.15266/) (2022)
- [Frontiers in Intelligent Colonoscopy](https://mlsanity.com/paper/2410.17241/) (2024)
- [From Easy to Hard: Progressive Active Learning Framework for Infrared Small Target Detection with Single Point Supervision](https://mlsanity.com/paper/2412.11154/) (2024)
- [Point-Teaching: Weakly Semi-Supervised Object Detection with Point Annotations](https://mlsanity.com/paper/2206.00274/) (2022)
- [Visual Recognition by Request](https://mlsanity.com/paper/2207.14227/) (2022)
- [SAP-DETR: Bridging the Gap Between Salient Points and Queries-Based Transformer Detector for Fast Model Convergency](https://mlsanity.com/paper/2211.02006/) (2022)
- [PointOBB-v3: Expanding Performance Boundaries of Single Point-Supervised Oriented Object Detection](https://mlsanity.com/paper/2501.13898/) (2025)
- [Graph Fusion Network for Multi-Oriented Object Detection](https://mlsanity.com/paper/2205.03562/) (2022)
- [Scaling Novel Object Detection with Weakly Supervised Detection Transformers](https://mlsanity.com/paper/2207.05205/) (2022)
- [Wholly-WOOD: Wholly Leveraging Diversified-quality Labels for Weakly-supervised Oriented Object Detection](https://mlsanity.com/paper/2502.09471/) (2025)
- [Semi-supervised Object Detection: A Survey on Recent Research and Progress](https://mlsanity.com/paper/2306.14106/) (2023)
- [What is Point Supervision Worth in Video Instance Segmentation?](https://mlsanity.com/paper/2404.01990/) (2024)
- [POLO -- Point-based, multi-class animal detection](https://mlsanity.com/paper/2410.11741/) (2024)
- [Meta-tuning Loss Functions and Data Augmentation for Few-shot Object Detection](https://mlsanity.com/paper/2304.12161/) (2023)
- [Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformer](https://mlsanity.com/paper/2407.08460/) (2024)
- [LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2503.18384/) (2025)

## Abstract (arXiv)

We propose a novel point annotated setting for the weakly semi-supervised object detection task, in which the dataset comprises small fully annotated images and large weakly annotated images by points. It achieves a balance between tremendous annotation burden and detection performance. Based on this setting, we analyze existing detectors and find that these detectors have difficulty in fully exploiting the power of the annotated points. To solve this, we introduce a new detector, Point DETR, which extends DETR by adding a point encoder. Extensive experiments conducted on MS-COCO dataset in various data settings show the effectiveness of our method. In particular, when using 20% fully labeled data from COCO, our detector achieves a promising performance, 33.3 AP, which outperforms a strong baseline (FCOS) by 2.0 AP, and we demonstrate the point annotations bring over 10 points in various AR metrics.