# Generalized Mask-aware IoU for Anchor Assignment for Real-time Instance Segmentation

arXiv: https://arxiv.org/abs/2312.17031 · HTML: https://mlsanity.com/paper/2312.17031/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 40 of its 49 references that are in the map.
- First posted on arXiv 28 Dec 2023.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection](https://mlsanity.com/paper/1912.02424/) (2019)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Distance-IoU Loss: Faster and Better Learning for Bounding Box Regression](https://mlsanity.com/paper/1911.08287/) (2019)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation](https://mlsanity.com/paper/2206.02777/) (2022)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [Boundary IoU: Improving Object-Centric Image Segmentation Evaluation](https://mlsanity.com/paper/2103.16562/) (2021)
- [Probabilistic Anchor Assignment with IoU Prediction for Object Detection](https://mlsanity.com/paper/2007.08103/) (2020)
- [CARAFE: Content-Aware ReAssembly of FEatures](https://mlsanity.com/paper/1905.02188/) (2019)
- [TensorMask: A Foundation for Dense Object Segmentation](https://mlsanity.com/paper/1903.12174/) (2019)
- [Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation](https://mlsanity.com/paper/2005.03572/) (2020)
- [YOLACT++: Better Real-time Instance Segmentation](https://mlsanity.com/paper/1912.06218/) (2019)
- [Sparse Instance Activation for Real-Time Instance Segmentation](https://mlsanity.com/paper/2203.12827/) (2022)
- [Recent Advances in Deep Learning for Object Detection](https://mlsanity.com/paper/1908.03673/) (2019)
- [Learning from Noisy Anchors for One-stage Object Detection](https://mlsanity.com/paper/1912.05086/) (2019)
- [Localization Recall Precision (LRP): A New Performance Metric for Object Detection](https://mlsanity.com/paper/1807.01696/) (2018)

## Abstract (arXiv)

This paper introduces Generalized Mask-aware Intersection-over-Union (GmaIoU) as a new measure for positive-negative assignment of anchor boxes during training of instance segmentation methods. Unlike conventional IoU measure or its variants, which only consider the proximity of anchor and ground-truth boxes; GmaIoU additionally takes into account the segmentation mask. This enables GmaIoU to provide more accurate supervision during training. We demonstrate the effectiveness of GmaIoU by replacing IoU with our GmaIoU in ATSS, a state-of-the-art (SOTA) assigner. Then, we train YOLACT, a real-time instance segmentation method, using our GmaIoU-based ATSS assigner. The resulting YOLACT based on the GmaIoU assigner outperforms (i) ATSS with IoU by ~ 1.0-1.5 mask AP, (ii) YOLACT with a fixed IoU threshold assigner by ~ 1.5-2 mask AP over different image sizes and (iii) decreases the inference time by 25 % owing to using less anchors. Taking advantage of this efficiency, we further devise GmaYOLACT, a faster and +7 mask AP points more accurate detector than YOLACT. Our best model achieves 38.7 mask AP at 26 fps on COCO test-dev establishing a new state-of-the-art for real-time instance segmentation.