mlsanityData to 7 Oct 2026

Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation

arXiv 2206.02777 ↗First posted 6 Jun 2022cs.CVcited by 341

Not on the map. Why?

Abstract (arXiv)

In this paper we present Mask DINO, a unified object detection and segmentation framework. Mask DINO extends DINO (DETR with Improved Denoising Anchor Boxes) by adding a mask prediction branch which supports all image segmentation tasks (instance, panoptic, and semantic). It makes use of the query embeddings from DINO to dot-product a high-resolution pixel embedding map to predict a set of binary masks. Some key components in DINO are extended for segmentation through a shared architecture and training process. Mask DINO is simple, efficient, and scalable, and it can benefit from joint large-scale detection and segmentation datasets. Our experiments show that Mask DINO significantly outperforms all existing specialized segmentation methods, both on a ResNet-50 backbone and a pre-trained model with SwinL backbone. Notably, Mask DINO establishes the best results to date on instance segmentation (54.5 AP on COCO), panoptic segmentation (59.4 PQ on COCO), and semantic segmentation (60.8 mIoU on ADE20K) among models under one billion parameters. Code is available at urlhttps://github.com/IDEACVR/MaskDINO.

Key facts. Cited by 341 papers in the map (data through 7 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 6 Jun 2022.

Builds on

35 of 47 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2017Mask R-CNNcited by 7,259
  9. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  10. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  11. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  12. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
23 more
  1. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  2. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  3. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  4. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  5. 2018Panoptic Segmentationcited by 796
  6. 2017Focal Loss for Dense Object Detectioncited by 749
  7. 2019Panoptic Feature Pyramid Networkscited by 644
  8. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 595
  9. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  10. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 486
  11. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  12. 2019PointRend: Image Segmentation as Renderingcited by 439
  13. 2021Conditional DETR for Fast Training Convergencecited by 422
  14. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  15. 2021End-to-End Semi-Supervised Object Detection with Soft Teachercited by 315
  16. 2021K-Net: Towards Unified Image Segmentationcited by 211
  17. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  18. 2021Instances as Queriescited by 157
  19. 2021Pointly-Supervised Instance Segmentationcited by 83
  20. 2021SeMask: Semantically Masked Transformers for Semantic Segmentationcited by 55
  21. 2021Simple Training Strategies and Model Scaling for Object Detectioncited by 25
  22. 2017ANCHOR: logically-centralized security for Software-Defined Networkscited by 8
  23. 2022Pyramid Fusion Transformer for Semantic Segmentationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 21; 2023: 92; 2024: 101; 2025: 81; 2026: 46.

341 papers.

Sort:

By area:

  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  2. 2022Planning-oriented Autonomous Drivingcited by 1,068
  3. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  4. 2023Segment Everything Everywhere All at Oncecited by 546
  5. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  6. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 387
  7. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  8. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
  9. 2023Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4Vcited by 291
  10. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
  11. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  12. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
38 more
  1. 2023Semantic-SAM: Segment and Recognize Anything at Any Granularitycited by 185
  2. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  3. 2022DETRs with Hybrid Matchingcited by 149
  4. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 136
  5. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  6. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  7. 2022Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learningcited by 115
  8. 2023The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open Worldcited by 114
  9. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  10. 2022Group DETR: Fast DETR Training with Group-Wise One-to-Many Assignmentcited by 96
  11. 2023OneFormer3D: One Transformer for Unified Point Cloud Segmentationcited by 95
  12. 2023Symphonize 3D Semantic Scene Completion with Contextual Instance Queriescited by 88
  13. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 85
  14. 2023Stronger, Fewer, & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentationcited by 82
  15. 2024Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactivelycited by 74
  16. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  17. 2024Improving 2D Feature Representations by 3D-Aware Fine-Tuningcited by 70
  18. 2025RF-DETR: Neural Architecture Search for Real-Time Detection Transformerscited by 70
  19. 2023AVSegFormer: Audio-Visual Segmentation with Transformercited by 66
  20. 2022Interactive Segmentation of Radiance Fieldscited by 60
  21. 2023UniHCP: A Unified Model for Human-Centric Perceptionscited by 59
  22. 2023A Survey on Open-Vocabulary Detection and Segmentation: Past, Present, and Futurecited by 59
  23. 2022Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Taskscited by 57
  24. 2024LaSagnA: Language-based Segmentation Assistant for Complex Queriescited by 55
  25. 2023Hierarchical Open-vocabulary Universal Image Segmentationcited by 53
  26. 2024Endora: Video Generation Models as Endoscopy Simulatorscited by 53
  27. 2023ISBNet: a 3D Point Cloud Instance Segmentation Network with Instance-aware Sampling and Box-aware Dynamic Convolutioncited by 52
  28. 2023Visual In-Context Promptingcited by 52
  29. 2022DiffusionInst: Diffusion Model for Instance Segmentationcited by 49
  30. 2022X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusioncited by 49
  31. 2023Rank-DETR for High Quality Object Detectioncited by 49
  32. 2024Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applicationscited by 49
  33. 2023Explicit Box Detection Unifies End-to-End Multi-Person Pose Estimationcited by 48
  34. 2023Mask-Attention-Free Transformer for 3D Instance Segmentationcited by 45
  35. 2023Detection Transformer with Stable Matchingcited by 43
  36. 2024Stream Query Denoising for Vectorized HD Map Constructioncited by 43
  37. 2023On Advantages of Mask-level Recognition for Outlier-aware Segmentationcited by 42
  38. 2023Open-Vocabulary Semantic Segmentation with Decoupled One-Pass Networkcited by 40

50 most cited of 341 All 341 (JSON)