# YOLACT++: Better Real-time Instance Segmentation

arXiv: https://arxiv.org/abs/1912.06218 · HTML: https://mlsanity.com/paper/1912.06218/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2019.

## Built on by (most cited first, 30 of 84)

- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection](https://mlsanity.com/paper/2103.05346/) (2021)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [Learning What Not to Segment: A New Perspective on Few-Shot Segmentation](https://mlsanity.com/paper/2203.07615/) (2022)
- [Crossover Learning for Fast Online Video Instance Segmentation](https://mlsanity.com/paper/2104.05970/) (2021)
- [Pointly-Supervised Instance Segmentation](https://mlsanity.com/paper/2104.06404/) (2021)
- [DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision](https://mlsanity.com/paper/2105.06464/) (2021)
- [OpenForensics: Large-Scale Challenging Dataset For Multi-Face Forgery Detection And Segmentation In-The-Wild](https://mlsanity.com/paper/2107.14480/) (2021)
- [NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions](https://mlsanity.com/paper/2202.12825/) (2022)
- [Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentation](https://mlsanity.com/paper/2104.05606/) (2021)
- [SOTR: Segmenting Objects with Transformers](https://mlsanity.com/paper/2108.06747/) (2021)
- [Box-supervised Instance Segmentation with Level Set Evolution](https://mlsanity.com/paper/2207.09055/) (2022)
- [You Only Segment Once: Towards Real-Time Panoptic Segmentation](https://mlsanity.com/paper/2303.14651/) (2023)
- [TF-Blender: Temporal Feature Blender for Video Object Detection](https://mlsanity.com/paper/2108.05821/) (2021)
- [FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation](https://mlsanity.com/paper/2303.08594/) (2023)
- [PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domain](https://mlsanity.com/paper/2306.04557/) (2023)
- [MSANet: Multi-Similarity and Attention Guidance for Boosting Few-Shot Segmentation](https://mlsanity.com/paper/2206.09667/) (2022)
- [SANeRF-HQ: Segment Anything for NeRF in High Quality](https://mlsanity.com/paper/2312.01531/) (2023)
- [Instance and Panoptic Segmentation Using Conditional Convolutions](https://mlsanity.com/paper/2102.03026/) (2021)
- [Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibration](https://mlsanity.com/paper/2307.00934/) (2023)
- [Panoptic Segmentation: A Review](https://mlsanity.com/paper/2111.10250/) (2021)
- [Real-time Instance Segmentation of Surgical Instruments using Attention and Multi-scale Feature Fusion](https://mlsanity.com/paper/2111.04911/) (2021)
- [Instance Segmentation for Autonomous Log Grasping in Forestry Operations](https://mlsanity.com/paper/2203.01902/) (2022)
- [Panoptic Segmentation Meets Remote Sensing](https://mlsanity.com/paper/2111.12126/) (2021)
- [Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review](https://mlsanity.com/paper/2203.10939/) (2022)
- [Trapped in texture bias? A large scale comparison of deep instance segmentation](https://mlsanity.com/paper/2401.09109/) (2024)
- [Recurrent Generic Contour-based Instance Segmentation with Progressive Learning](https://mlsanity.com/paper/2301.08898/) (2023)
- [CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2311.00987/) (2023)
- [Benchmarking the Robustness of Instance Segmentation Models](https://mlsanity.com/paper/2109.01123/) (2021)
- [Instance As Identity: A Generic Online Paradigm for Video Instance Segmentation](https://mlsanity.com/paper/2208.03079/) (2022)

## Abstract (arXiv)

We present a simple, fully-convolutional model for real-time (>30 fps) instance segmentation that achieves competitive results on MS COCO evaluated on a single Titan Xp, which is significantly faster than any previous state-of-the-art approach. Moreover, we obtain this result after training on only one GPU. We accomplish this by breaking instance segmentation into two parallel subtasks: (1) generating a set of prototype masks and (2) predicting per-instance mask coefficients. Then we produce instance masks by linearly combining the prototypes with the mask coefficients. We find that because this process doesn't depend on repooling, this approach produces very high-quality masks and exhibits temporal stability for free. Furthermore, we analyze the emergent behavior of our prototypes and show they learn to localize instances on their own in a translation variant manner, despite being fully-convolutional. We also propose Fast NMS, a drop-in 12 ms faster replacement for standard NMS that only has a marginal performance penalty. Finally, by incorporating deformable convolutions into the backbone network, optimizing the prediction head with better anchor scales and aspect ratios, and adding a novel fast mask re-scoring branch, our YOLACT++ model can achieve 34.1 mAP on MS COCO at 33.5 fps, which is fairly close to the state-of-the-art approaches while still running at real-time.