# Mask-aware IoU for Anchor Assignment in Real-time Instance Segmentation

arXiv: https://arxiv.org/abs/2110.09734 · HTML: https://mlsanity.com/paper/2110.09734/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 45 references that are in the map.
- First posted on arXiv 19 Oct 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [Boundary IoU: Improving Object-Centric Image Segmentation Evaluation](https://mlsanity.com/paper/2103.16562/) (2021)
- [Probabilistic Anchor Assignment with IoU Prediction for Object Detection](https://mlsanity.com/paper/2007.08103/) (2020)
- [CARAFE: Content-Aware ReAssembly of FEatures](https://mlsanity.com/paper/1905.02188/) (2019)
- [TensorMask: A Foundation for Dense Object Segmentation](https://mlsanity.com/paper/1903.12174/) (2019)
- [Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation](https://mlsanity.com/paper/2005.03572/) (2020)
- [YOLACT++: Better Real-time Instance Segmentation](https://mlsanity.com/paper/1912.06218/) (2019)
- [Learning from Noisy Anchors for One-stage Object Detection](https://mlsanity.com/paper/1912.05086/) (2019)
- [Localization Recall Precision (LRP): A New Performance Metric for Object Detection](https://mlsanity.com/paper/1807.01696/) (2018)
- [CenterMask: single shot instance segmentation with point representation](https://mlsanity.com/paper/2004.04446/) (2020)
- [Multiple Anchor Learning for Visual Object Detection](https://mlsanity.com/paper/1912.02252/) (2019)
- [PolarMask++: Enhanced Polar Representation for Single-Shot Instance Segmentation and Beyond](https://mlsanity.com/paper/2105.02184/) (2021)
- [RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free](https://mlsanity.com/paper/1901.03353/) (2019)
- [Rank & Sort Loss for Object Detection and Instance Segmentation](https://mlsanity.com/paper/2107.11669/) (2021)

## Built on by (most cited first, 5 of 5)

- [Sparse Instance Activation for Real-Time Instance Segmentation](https://mlsanity.com/paper/2203.12827/) (2022)
- [MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection](https://mlsanity.com/paper/2309.14976/) (2023)
- [Bucketed Ranking-based Losses for Efficient Training of Object Detectors](https://mlsanity.com/paper/2407.14204/) (2024)
- [Real-time Automatic M-mode Echocardiography Measurement with Panel Attention from Local-to-Global Pixels](https://mlsanity.com/paper/2308.07717/) (2023)
- [Generalized Mask-aware IoU for Anchor Assignment for Real-time Instance Segmentation](https://mlsanity.com/paper/2312.17031/) (2023)

## Abstract (arXiv)

This paper presents Mask-aware Intersection-over-Union (maIoU) for assigning anchor boxes as positives and negatives during training of instance segmentation methods. Unlike conventional IoU or its variants, which only considers the proximity of two boxes; maIoU consistently measures the proximity of an anchor box with not only a ground truth box but also its associated ground truth mask. Thus, additionally considering the mask, which, in fact, represents the shape of the object, maIoU enables a more accurate supervision during training. We present the effectiveness of maIoU on a state-of-the-art (SOTA) assigner, ATSS, by replacing IoU operation by our maIoU and training YOLACT, a SOTA real-time instance segmentation method. Using ATSS with maIoU consistently outperforms (i) ATSS with IoU by ~ 1 mask AP, (ii) baseline YOLACT with fixed IoU threshold assigner by ~ 2 mask AP over different image sizes and (iii) decreases the inference time by 25 % owing to using less anchors. Then, exploiting this efficiency, we devise maYOLACT, a faster and +6 AP more accurate detector than YOLACT. Our best model achieves 37.7 mask AP at 25 fps on COCO test-dev establishing a new state-of-the-art for real-time instance segmentation. Code is available at https://github.com/kemaloksuz/Mask-aware-IoU