# FastInst: A Simple Query-Based Model for Real-Time Instance Segmentation

arXiv: https://arxiv.org/abs/2303.08594 · HTML: https://mlsanity.com/paper/2303.08594/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 50 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [Bag of Tricks for Image Classification with Convolutional Neural Networks](https://mlsanity.com/paper/1812.01187/) (2018)
- [MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers](https://mlsanity.com/paper/2012.00759/) (2020)
- [Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation](https://mlsanity.com/paper/2206.02777/) (2022)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)

## Built on by (most cited first, 30 of 32)

- [RF-DETR: Neural Architecture Search for Real-Time Detection Transformers](https://mlsanity.com/paper/2511.09554/) (2025)
- [Unifying 3D Vision-Language Understanding via Promptable Queries](https://mlsanity.com/paper/2405.11442/) (2024)
- [VST++: Efficient and Stronger Visual Saliency Transformer](https://mlsanity.com/paper/2310.11725/) (2023)
- [DFormer: Diffusion-guided Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2306.03437/) (2023)
- [AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generation](https://mlsanity.com/paper/2501.09503/) (2025)
- [ForestFormer3D: A Unified Framework for End-to-End Segmentation of Forest LiDAR 3D Point Clouds](https://mlsanity.com/paper/2506.16991/) (2025)
- [VLTP: Vision-Language Guided Token Pruning for Task-Oriented Segmentation](https://mlsanity.com/paper/2409.08464/) (2024)
- [SBCFormer: Lightweight Network Capable of Full-size ImageNet Classification at 1 FPS on Single Board Computers](https://mlsanity.com/paper/2311.03747/) (2023)
- [MobileInst: Video Instance Segmentation on the Mobile](https://mlsanity.com/paper/2303.17594/) (2023)
- [LLaPa: A Vision-Language Model Framework for Counterfactual-Aware Procedural Planning](https://mlsanity.com/paper/2507.08496/) (2025)
- [Refining Context-Entangled Content Segmentation via Curriculum Selection and Anti-Curriculum Promotion](https://mlsanity.com/paper/2602.01183/) (2026)
- [Extreme Point Supervised Instance Segmentation](https://mlsanity.com/paper/2405.20729/) (2024)
- [QORT-Former: Query-optimized Real-time Transformer for Understanding Two Hands Manipulating Objects](https://mlsanity.com/paper/2502.19769/) (2025)
- [LaSSM: Efficient Semantic-Spatial Query Decoding via Local Aggregation and State Space Models for 3D Instance Segmentation](https://mlsanity.com/paper/2602.11007/) (2026)
- [EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation](https://mlsanity.com/paper/2603.18739/) (2026)
- [GMT: Guided Mask Transformer for Leaf Instance Segmentation](https://mlsanity.com/paper/2406.17109/) (2024)
- [GAInS: Gradient Anomaly-aware Biomedical Instance Segmentation](https://mlsanity.com/paper/2409.13988/) (2024)
- [DI-MaskDINO: A Joint Object Detection and Instance Segmentation Model](https://mlsanity.com/paper/2410.16707/) (2024)
- [EM3M: An Electron Micrograph Dataset for Microstructural Segmentation and Generation](https://mlsanity.com/paper/2508.16239/) (2025)
- [Channel-wise Motion Features for Efficient Motion Segmentation](https://mlsanity.com/paper/2507.13082/) (2025)
- [IAUNet: Instance-Aware U-Net](https://mlsanity.com/paper/2508.01928/) (2025)
- [MambaTrans: Multimodal Fusion Image Translation via Large Language Model Priors for Downstream Visual Tasks](https://mlsanity.com/paper/2508.07803/) (2025)
- [UniC-Lift: Unified 3D Instance Segmentation via Contrastive Learning](https://mlsanity.com/paper/2512.24763/) (2025)
- [Integrated Image-Text Based on Semi-supervised Learning for Small Sample Instance Segmentation](https://mlsanity.com/paper/2410.16063/) (2024)
- [FastInstShadow: A Simple Query-Based Model for Instance Shadow Detection](https://mlsanity.com/paper/2503.07517/) (2025)
- [WaveInst: A Frequency-Domain Enhanced Network for Fine-Grained Thin Tree Trunk Extraction in Forest Scenes](https://mlsanity.com/paper/2505.01656/) (2025)
- [O2Former:Direction-Aware and Multi-Scale Query Enhancement for SAR Ship Instance Segmentation](https://mlsanity.com/paper/2506.11913/) (2025)
- [SPDA-SAM: A Self-prompted Depth-Aware Segment Anything Model for Instance Segmentation](https://mlsanity.com/paper/2602.06335/) (2026)
- [A Turbo-Inference Strategy for Object Detection and Instance Segmentation](https://mlsanity.com/paper/2606.12371/) (2026)
- [SAM-MT: Real-Time Interactive Multi-Target Video Segmentation](https://mlsanity.com/paper/2607.08688/) (2026)

## Abstract (arXiv)

Recent attention in instance segmentation has focused on query-based models. Despite being non-maximum suppression (NMS)-free and end-to-end, the superiority of these models on high-accuracy real-time benchmarks has not been well demonstrated. In this paper, we show the strong potential of query-based models on efficient instance segmentation algorithm designs. We present FastInst, a simple, effective query-based framework for real-time instance segmentation. FastInst can execute at a real-time speed (i.e., 32.5 FPS) while yielding an AP of more than 40 (i.e., 40.5 AP) on COCO test-dev without bells and whistles. Specifically, FastInst follows the meta-architecture of recently introduced Mask2Former. Its key designs include instance activation-guided queries, dual-path update strategy, and ground truth mask-guided learning, which enable us to use lighter pixel decoders, fewer Transformer decoder layers, while achieving better performance. The experiments show that FastInst outperforms most state-of-the-art real-time counterparts, including strong fully convolutional baselines, in both speed and accuracy. Code can be found at https://github.com/junjiehe96/FastInst .