mlsanityData to 7 Oct 2026

Panoptic Segmentation

arXiv 1801.00868 ↗First posted 3 Jan 2018cs.CVcited by 796

Not on the map. Why?

Cited by 25 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose and study a task we name panoptic segmentation (PS). Panoptic segmentation unifies the typically distinct tasks of semantic segmentation (assign a class label to each pixel) and instance segmentation (detect and segment each object instance). The proposed task requires generating a coherent scene segmentation that is rich and complete, an important step toward real-world vision systems. While early work in computer vision addressed related image/scene parsing tasks, these are not currently popular, possibly due to lack of appropriate metrics or associated recognition challenges. To address this, we propose a novel panoptic quality (PQ) metric that captures performance for all classes (stuff and things) in an interpretable and unified manner. Using the proposed metric, we perform a rigorous study of both human and machine performance for PS on three existing datasets, revealing interesting insights about the task. The aim of our work is to revive the interest of the community in a more unified view of image segmentation.

Key facts. Cited by 796 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jan 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 119; 2022: 130; 2023: 201; 2024: 151; 2025: 117; 2026: 77.

796 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,158
  2. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  3. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  4. 2022Planning-oriented Autonomous Drivingcited by 1,068
  5. 2021Transformers in Vision: A Surveycited by 1,009
  6. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  7. 2023Segment Everything Everywhere All at Oncecited by 546
  8. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  9. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  10. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  11. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  12. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
38 more
  1. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  2. 2023Fast Segment Anythingcited by 286
  3. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
  4. 2023Personalize Segment Anything Model with One Shotcited by 250
  5. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  6. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 215
  7. 2021Concealed Object Detectioncited by 212
  8. 2021K-Net: Towards Unified Image Segmentationcited by 211
  9. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
  10. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  11. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  12. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  13. 2022Metrics reloaded: Recommendations for image analysis validationcited by 190
  14. 2021Boundary IoU: Improving Object-Centric Image Segmentation Evaluationcited by 186
  15. 2023Semantic-SAM: Segment and Recognize Anything at Any Granularitycited by 185
  16. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  17. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  18. 2021SceneGraphFusion: Incremental 3D Scene Graph Prediction from RGB-D Sequencescited by 159
  19. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  20. 2022Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentationcited by 152
  21. 2021Camouflaged Object Segmentation with Distraction Miningcited by 151
  22. 2022FlexiViT: One Model for All Patch Sizescited by 151
  23. 2022Benchmarking Self-Supervised Learning on Diverse Pathology Datasetscited by 148
  24. 2021Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphscited by 144
  25. 2021InstanceRefer: Cooperative Holistic Understanding for Visual Grounding on Point Clouds through Instance Multi-level Contextual Referringcited by 140
  26. 2022Open-Vocabulary Universal Image Segmentation with MaskCLIPcited by 131
  27. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 126
  28. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 126
  29. 2021MSeg: A Composite Dataset for Multi-domain Semantic Segmentationcited by 122
  30. 2023Aligning Bag of Regions for Open-Vocabulary Object Detectioncited by 119
  31. 2021A Survey of Visual Transformerscited by 117
  32. 2022Panoptic Scene Graph Generationcited by 114
  33. 2023PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentationcited by 114
  34. 2023The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open Worldcited by 114
  35. 2023CLIPSelf: Vision Transformer Distills Itself for Open-Vocabulary Dense Predictioncited by 114
  36. 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
  37. 2021Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformerscited by 109
  38. 2023CellViT: Vision Transformers for Precise Cell Segmentation and Classificationcited by 108

50 most cited of 796 All 796 (JSON)