# Mask Transfiner for High-Quality Instance Segmentation

arXiv: https://arxiv.org/abs/2111.13673 · HTML: https://mlsanity.com/paper/2111.13673/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 51 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [Path Aggregation Network for Instance Segmentation](https://mlsanity.com/paper/1803.01534/) (2018)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019)
- [Panoptic-DeepLab](https://mlsanity.com/paper/1910.04751/) (2019)
- [SOLO: Segmenting Objects by Locations](https://mlsanity.com/paper/1912.04488/) (2019)
- [Mask Scoring R-CNN](https://mlsanity.com/paper/1903.00241/) (2019)
- [K-Net: Towards Unified Image Segmentation](https://mlsanity.com/paper/2106.14855/) (2021)
- [CenterMask : Real-Time Anchor-Free Instance Segmentation](https://mlsanity.com/paper/1911.06667/) (2019)
- [PolarMask: Single Shot Instance Segmentation with Polar Representation](https://mlsanity.com/paper/1909.13226/) (2019)
- [Boundary IoU: Improving Object-Centric Image Segmentation Evaluation](https://mlsanity.com/paper/2103.16562/) (2021)
- [Gated-SCNN: Gated Shape CNNs for Semantic Segmentation](https://mlsanity.com/paper/1907.05740/) (2019)
- [Instances as Queries](https://mlsanity.com/paper/2105.01928/) (2021)
- [BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation](https://mlsanity.com/paper/2001.00309/) (2020)

## Built on by (most cited first, 30 of 64)

- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [DynIBaR: Neural Dynamic Image-Based Rendering](https://mlsanity.com/paper/2211.11082/) (2022)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [High-Quality Entity Segmentation](https://mlsanity.com/paper/2211.05776/) (2022)
- [Hi4D: 4D Instance Segmentation of Close Human Interaction](https://mlsanity.com/paper/2303.15380/) (2023)
- [LVOS: A Benchmark for Long-term Video Object Segmentation](https://mlsanity.com/paper/2211.10181/) (2022)
- [LVOS: A Benchmark for Large-scale Long-term Video Object Segmentation](https://mlsanity.com/paper/2404.19326/) (2024)
- [Vision Transformers with Mixed-Resolution Tokenization](https://mlsanity.com/paper/2304.00287/) (2023)
- [AISFormer: Amodal Instance Segmentation with Transformer](https://mlsanity.com/paper/2210.06323/) (2022)
- [OSFormer: One-Stage Camouflaged Instance Segmentation with Transformers](https://mlsanity.com/paper/2207.02255/) (2022)
- [SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Process](https://mlsanity.com/paper/2312.12425/) (2023)
- [Video Mask Transfiner for High-Quality Video Instance Segmentation](https://mlsanity.com/paper/2207.14012/) (2022)
- [The Devil is in the Points: Weakly Semi-Supervised Instance Segmentation via Point-Guided Mask Representation](https://mlsanity.com/paper/2303.15062/) (2023)
- [Mask-Free Video Instance Segmentation](https://mlsanity.com/paper/2303.15904/) (2023)
- [SAMRefiner: Taming Segment Anything Model for Universal Mask Refinement](https://mlsanity.com/paper/2502.06756/) (2025)
- [DFormer: Diffusion-guided Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2306.03437/) (2023)
- [Cascade-DETR: Delving into High-Quality Universal Object Detection](https://mlsanity.com/paper/2307.11035/) (2023)
- [Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Model](https://mlsanity.com/paper/2406.19369/) (2024)
- [BoxSnake: Polygonal Instance Segmentation with Box Supervision](https://mlsanity.com/paper/2303.11630/) (2023)
- [SHaRPose: Sparse High-Resolution Representation for Human Pose Estimation](https://mlsanity.com/paper/2312.10758/) (2023)
- [DynaMask: Dynamic Mask Selection for Instance Segmentation](https://mlsanity.com/paper/2303.07868/) (2023)
- [CalibNet: Dual-branch Cross-modal Calibration for RGB-D Salient Instance Segmentation](https://mlsanity.com/paper/2307.08098/) (2023)
- [Scene-Generalizable Interactive Segmentation of Radiance Fields](https://mlsanity.com/paper/2308.05104/) (2023)
- [CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2311.00987/) (2023)
- [Empowering DINO Representations for Underwater Instance Segmentation via Aligner and Prompter](https://mlsanity.com/paper/2511.08334/) (2025)
- [PatchDCT: Patch Refinement for High Quality Instance Segmentation](https://mlsanity.com/paper/2302.02693/) (2023)
- [Deblurring Masked Autoencoder is Better Recipe for Ultrasound Image Recognition](https://mlsanity.com/paper/2306.08249/) (2023)
- [Video Task Decathlon: Unifying Image and Video Tasks in Autonomous Driving](https://mlsanity.com/paper/2309.04422/) (2023)
- [EviPrompt: A Training-Free Evidential Prompt Generation Method for Segment Anything Model in Medical Images](https://mlsanity.com/paper/2311.06400/) (2023)
- [Adaptive Human Matting for Dynamic Videos](https://mlsanity.com/paper/2304.06018/) (2023)

## Abstract (arXiv)

Two-stage and query-based instance segmentation methods have achieved remarkable results. However, their segmented masks are still very coarse. In this paper, we present Mask Transfiner for high-quality and efficient instance segmentation. Instead of operating on regular dense tensors, our Mask Transfiner decomposes and represents the image regions as a quadtree. Our transformer-based approach only processes detected error-prone tree nodes and self-corrects their errors in parallel. While these sparse pixels only constitute a small proportion of the total number, they are critical to the final mask quality. This allows Mask Transfiner to predict highly accurate instance masks, at a low computational cost. Extensive experiments demonstrate that Mask Transfiner outperforms current instance segmentation methods on three popular benchmarks, significantly improving both two-stage and query-based frameworks by a large margin of +3.0 mask AP on COCO and BDD100K, and +6.6 boundary AP on Cityscapes. Our code and trained models will be available at http://vis.xyz/pub/transfiner.