# Multiple Anchor Learning for Visual Object Detection

arXiv: https://arxiv.org/abs/1912.02252 · HTML: https://mlsanity.com/paper/1912.02252/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Dec 2019.

## Built on by (most cited first, 29 of 29)

- [TOOD: Task-aligned One-stage Object Detection](https://mlsanity.com/paper/2108.07755/) (2021)
- [Probabilistic two-stage detection](https://mlsanity.com/paper/2103.07461/) (2021)
- [Consistent-Teacher: Towards Reducing Inconsistent Pseudo-targets in Semi-supervised Object Detection](https://mlsanity.com/paper/2209.01589/) (2022)
- [A Dual Weighting Label Assignment Scheme for Object Detection](https://mlsanity.com/paper/2203.09730/) (2022)
- [MonoLSS: Learnable Sample Selection For Monocular 3D Detection](https://mlsanity.com/paper/2312.14474/) (2023)
- [Improving Object Detection by Label Assignment Distillation](https://mlsanity.com/paper/2108.10520/) (2021)
- [Learning to Filter: Siamese Relation Network for Robust Tracking](https://mlsanity.com/paper/2104.00829/) (2021)
- [ObjectBox: From Centers to Boxes for Anchor-Free Object Detection](https://mlsanity.com/paper/2207.06985/) (2022)
- [Mutual Supervision for Dense Object Detection](https://mlsanity.com/paper/2109.05986/) (2021)
- [Continuous Human Action Recognition for Human-Machine Interaction: A Review](https://mlsanity.com/paper/2202.13096/) (2022)
- [AGAR a microbial colony dataset for deep learning detection](https://mlsanity.com/paper/2108.01234/) (2021)
- [Dynamic Label Assignment for Object Detection by Combining Predicted IoUs and Anchor IoUs](https://mlsanity.com/paper/2201.09396/) (2022)
- [Deep Learning Approaches for Seizure Video Analysis: A Review](https://mlsanity.com/paper/2312.10930/) (2023)
- [Anchor Pruning for Object Detection](https://mlsanity.com/paper/2104.00432/) (2021)
- [PDNet: Toward Better One-Stage Object Detection With Prediction Decoupling](https://mlsanity.com/paper/2104.13876/) (2021)
- [Mask-aware IoU for Anchor Assignment in Real-time Instance Segmentation](https://mlsanity.com/paper/2110.09734/) (2021)
- [Revisiting AP Loss for Dense Object Detection: Adaptive Ranking Pair Selection](https://mlsanity.com/paper/2207.12042/) (2022)
- [Gaussian Guided IoU: A Better Metric for Balanced Learning on Object Detection](https://mlsanity.com/paper/2103.13613/) (2021)
- [Modulating Localization and Classification for Harmonized Object Detection](https://mlsanity.com/paper/2103.08958/) (2021)
- [GUTS: Generalized Uncertainty-Aware Thompson Sampling for Multi-Agent Active Search](https://mlsanity.com/paper/2304.02075/) (2023)
- [Deep Omni-supervised Learning for Rib Fracture Detection from Chest Radiology Images](https://mlsanity.com/paper/2306.13301/) (2023)
- [Density-based Object Detection in Crowded Scenes](https://mlsanity.com/paper/2504.09819/) (2025)
- [Progressive Hard-case Mining across Pyramid Levels for Object Detection](https://mlsanity.com/paper/2109.07217/) (2021)
- [Discovery-and-Selection: Towards Optimal Multiple Instance Learning for Weakly Supervised Object Detection](https://mlsanity.com/paper/2110.09060/) (2021)
- [HR-RCNN: Hierarchical Relational Reasoning for Object Detection](https://mlsanity.com/paper/2110.13892/) (2021)
- [Toward Minimal Misalignment at Minimal Cost in One-Stage and Anchor-Free Object Detection](https://mlsanity.com/paper/2112.08902/) (2021)
- [Multi-Semantic Interactive Learning for Object Detection](https://mlsanity.com/paper/2303.10411/) (2023)
- [Generalized Mask-aware IoU for Anchor Assignment for Real-time Instance Segmentation](https://mlsanity.com/paper/2312.17031/) (2023)
- [Real-Time 3D Object Detection with Inference-Aligned Learning](https://mlsanity.com/paper/2511.16140/) (2025)

## Abstract (arXiv)

Classification and localization are two pillars of visual object detectors. However, in CNN-based detectors, these two modules are usually optimized under a fixed set of candidate (or anchor) bounding boxes. This configuration significantly limits the possibility to jointly optimize classification and localization. In this paper, we propose a Multiple Instance Learning (MIL) approach that selects anchors and jointly optimizes the two modules of a CNN-based object detector. Our approach, referred to as Multiple Anchor Learning (MAL), constructs anchor bags and selects the most representative anchors from each bag. Such an iterative selection process is potentially NP-hard to optimize. To address this issue, we solve MAL by repetitively depressing the confidence of selected anchors by perturbing their corresponding features. In an adversarial selection-depression manner, MAL not only pursues optimal solutions but also fully leverages multiple anchors/features to learn a detection model. Experiments show that MAL improves the baseline RetinaNet with significant margins on the commonly used MS-COCO object detection benchmark and achieves new state-of-the-art detection performance compared with recent methods.