mlsanityData to 14 Oct 2026

Masked-attention Mask Transformer for Universal Image Segmentation

arXiv 2112.01527 ↗First posted 2 Dec 2021cs.CV cs.AI cs.LGcited by 2,351

Not on the map. Why?

Cited by 120 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Image segmentation is about grouping pixels with different semantics, e.g., category or instance membership, where each choice of semantics defines a task. While only the semantics of each task differ, current research focuses on designing specialized architectures for each task. We present Masked-attention Mask Transformer (Mask2Former), a new architecture capable of addressing any image segmentation task (panoptic, instance or semantic). Its key components include masked attention, which extracts localized features by constraining cross-attention within predicted mask regions. In addition to reducing the research effort by at least three times, it outperforms the best specialized architectures by a significant margin on four popular datasets. Most notably, Mask2Former sets a new state-of-the-art for panoptic segmentation (57.8 PQ on COCO), instance segmentation (50.1 AP on COCO) and semantic segmentation (57.7 mIoU on ADE20K).

Key facts. Cited by 2,351 papers in the map (data through 14 Oct 2026). Builds on 56 of its 66 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

56 of 66 references.

References · 56

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  8. 2020End-to-End Object Detection with Transformerscited by 7,414
  9. 2017Mask R-CNNcited by 7,277
  10. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  11. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  12. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
38 more
  1. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  2. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  3. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  4. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  5. 2016Pyramid Scene Parsing Networkcited by 2,479
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  7. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  8. 2017Non-local Neural Networkscited by 2,293
  9. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  10. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  11. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  12. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  13. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  14. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  15. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  16. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,020
  17. 2018Dual Attention Network for Scene Segmentationcited by 1,003
  18. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  19. 2018Panoptic Segmentationcited by 802
  20. 2017Focal Loss for Dense Object Detectioncited by 752
  21. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  22. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 718
  23. 2019Panoptic Feature Pyramid Networkscited by 645
  24. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
  25. 2019Hybrid Task Cascade for Instance Segmentationcited by 481
  26. 2019PointRend: Image Segmentation as Renderingcited by 440
  27. 2021Conditional DETR for Fast Training Convergencecited by 424
  28. 2019Panoptic-DeepLabcited by 363
  29. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  30. 2020Conditional Convolutions for Instance Segmentationcited by 314
  31. 2021K-Net: Towards Unified Image Segmentationcited by 212
  32. 2021Boundary IoU: Improving Object-Centric Image Segmentation Evaluationcited by 188
  33. 2020Hierarchical Multi-Scale Attention for Semantic Segmentationcited by 183
  34. 2021Fast Convergence of DETR with Spatially Modulated Co-Attentioncited by 176
  35. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  36. 2021Instances as Queriescited by 158
  37. 2020Rethinking Transformer-based Set Prediction for Object Detectioncited by 156
  38. 2019YOLACT++: Better Real-time Instance Segmentationcited by 84

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 154; 2023: 514; 2024: 590; 2025: 601; 2026: 489.

2,351 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,234
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  4. 2024Depth Anything V2cited by 1,422
  5. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  6. 2022Planning-oriented Autonomous Drivingcited by 1,087
  7. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  8. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  9. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  10. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  11. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  12. 2023Segment Everything Everywhere All at Oncecited by 548
38 more
  1. 2023LERF: Language Embedded Radiance Fieldscited by 543
  2. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  3. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
  4. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  5. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  6. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  7. 2022Point Transformer V2: Grouped Vector Attention and Partition-based Poolingcited by 388
  8. 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
  9. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  10. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 344
  11. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  12. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  13. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 318
  14. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  15. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  16. 2022Mask3D: Mask Transformer for 3D Semantic Instance Segmentationcited by 303
  17. 2023Side Adapter Network for Open-Vocabulary Semantic Segmentationcited by 300
  18. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  19. 2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
  20. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
  21. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  22. 2023Personalize Segment Anything Model with One Shotcited by 250
  23. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 241
  24. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 239
  25. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 236
  26. 2024Rotary Position Embedding for Vision Transformercited by 236
  27. 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 231
  28. 2022Towards Robust Blind Face Restoration with Codebook Lookup Transformercited by 231
  29. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  30. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  31. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  32. 2022Visual Attention Networkcited by 210
  33. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 201
  34. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 198
  35. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  36. 2023SegGPT: Segmenting Everything In Contextcited by 194
  37. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 193
  38. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192

50 most cited of 2,351 All 2,351 (JSON)