# Sparse Instance Activation for Real-Time Instance Segmentation

arXiv: https://arxiv.org/abs/2203.12827 · HTML: https://mlsanity.com/paper/2203.12827/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 52 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 30 of 44)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://mlsanity.com/paper/1812.01187/) (2018)
- [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [K-Net: Towards Unified Image Segmentation](https://mlsanity.com/paper/2106.14855/) (2021)
- [CenterMask : Real-Time Anchor-Free Instance Segmentation](https://mlsanity.com/paper/1911.06667/) (2019)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [End-to-end people detection in crowded scenes](https://mlsanity.com/paper/1506.04878/) (2015)

## Built on by (most cited first, 30 of 63)

- [RTMDet: An Empirical Study of Designing Real-Time Object Detectors](https://mlsanity.com/paper/2212.07784/) (2022)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [Superpoint Transformer for 3D Scene Instance Segmentation](https://mlsanity.com/paper/2211.15766/) (2022)
- [MGMap: Mask-Guided Learning for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2404.00876/) (2024)
- [LATR: 3D Lane Detection from Monocular Images with Transformer](https://mlsanity.com/paper/2308.04583/) (2023)
- [You Only Segment Once: Towards Real-Time Panoptic Segmentation](https://mlsanity.com/paper/2303.14651/) (2023)
- [FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation](https://mlsanity.com/paper/2303.08594/) (2023)
- [CLUSTSEG: Clustering for Universal Segmentation](https://mlsanity.com/paper/2305.02187/) (2023)
- [BoxTeacher: Exploring High-Quality Pseudo Labels for Weakly Supervised Instance Segmentation](https://mlsanity.com/paper/2210.05174/) (2022)
- [UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Images](https://mlsanity.com/paper/2402.16663/) (2024)
- [Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentation](https://mlsanity.com/paper/2412.04533/) (2024)
- [DFormer: Diffusion-guided Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2306.03437/) (2023)
- [Learning Equivariant Segmentation with Instance-Unique Querying](https://mlsanity.com/paper/2210.00911/) (2022)
- [Towards Open-World Segmentation of Parts](https://mlsanity.com/paper/2305.16804/) (2023)
- [3D-QueryIS: A Query-based Framework for 3D Instance Segmentation](https://mlsanity.com/paper/2211.09375/) (2022)
- [Panoptic-FlashOcc: An Efficient Baseline to Marry Semantic Occupancy with Panoptic via Instance Center](https://mlsanity.com/paper/2406.10527/) (2024)
- [BuildMapper: A Fully Learnable Framework for Vectorized Building Contour Extraction](https://mlsanity.com/paper/2211.03373/) (2022)
- [CalibNet: Dual-branch Cross-modal Calibration for RGB-D Salient Instance Segmentation](https://mlsanity.com/paper/2307.08098/) (2023)
- [USIS16K: High-Quality Dataset for Underwater Salient Instance Segmentation](https://mlsanity.com/paper/2506.19472/) (2025)
- [DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representation](https://mlsanity.com/paper/2406.16072/) (2024)
- [A Unified Query-based Paradigm for Camouflaged Instance Segmentation](https://mlsanity.com/paper/2308.07392/) (2023)
- [AIM 2025 Rip Current Segmentation (RipSeg) Challenge Report](https://mlsanity.com/paper/2508.13401/) (2025)
- [MobileInst: Video Instance Segmentation on the Mobile](https://mlsanity.com/paper/2303.17594/) (2023)
- [ClusterFormer: Clustering As A Universal Visual Learner](https://mlsanity.com/paper/2309.13196/) (2023)
- [Improving Hierarchical Representations of Vectorized HD Maps with Perspective Clues](https://mlsanity.com/paper/2404.11155/) (2024)
- [Real-time GeoAI for High-resolution Mapping and Segmentation of Arctic Permafrost Features](https://mlsanity.com/paper/2306.05341/) (2023)
- [VIPriors 3: Visual Inductive Priors for Data-Efficient Deep Learning Challenges](https://mlsanity.com/paper/2305.19688/) (2023)
- [Interaction-aware Representation Modeling with Co-occurrence Consistency for Egocentric Hand-Object Parsing](https://mlsanity.com/paper/2602.20597/) (2026)
- [Extreme Point Supervised Instance Segmentation](https://mlsanity.com/paper/2405.20729/) (2024)
- [QORT-Former: Query-optimized Real-time Transformer for Understanding Two Hands Manipulating Objects](https://mlsanity.com/paper/2502.19769/) (2025)

## Abstract (arXiv)

In this paper, we propose a conceptually novel, efficient, and fully convolutional framework for real-time instance segmentation. Previously, most instance segmentation methods heavily rely on object detection and perform mask prediction based on bounding boxes or dense centers. In contrast, we propose a sparse set of instance activation maps, as a new object representation, to highlight informative regions for each foreground object. Then instance-level features are obtained by aggregating features according to the highlighted regions for recognition and segmentation. Moreover, based on bipartite matching, the instance activation maps can predict objects in a one-to-one style, thus avoiding non-maximum suppression (NMS) in post-processing. Owing to the simple yet effective designs with instance activation maps, SparseInst has extremely fast inference speed and achieves 40 FPS and 37.9 AP on the COCO benchmark, which significantly outperforms the counterparts in terms of speed and accuracy. Code and models are available at https://github.com/hustvl/SparseInst.