mlsanityData to 7 Oct 2026

K-Net: Towards Unified Image Segmentation

arXiv 2106.14855 ↗First posted 28 Jun 2021cs.CV cs.AIcited by 211

Not on the map. Why?

Abstract (arXiv)

Semantic, instance, and panoptic segmentations have been addressed using different and specialized frameworks despite their underlying connections. This paper presents a unified, simple, and effective framework for these essentially similar tasks. The framework, named K-Net, segments both instances and semantic categories consistently by a group of learnable kernels, where each kernel is responsible for generating a mask for either a potential instance or a stuff class. To remedy the difficulties of distinguishing various instances, we propose a kernel update strategy that enables each kernel dynamic and conditional on its meaningful group in the input image. K-Net can be trained in an end-to-end manner with bipartite matching, and its training and inference are naturally NMS-free and box-free. Without bells and whistles, K-Net surpasses all previous published state-of-the-art single-model results of panoptic segmentation on MS COCO test-dev split and semantic segmentation on ADE20K val split with 55.2% PQ and 54.3% mIoU, respectively. Its instance segmentation performance is also on par with Cascade Mask R-CNN on MS COCO with 60%-90% faster inference speeds. Code and models will be released at https://github.com/ZwwWayne/K-Net/.

Key facts. Cited by 211 papers in the map (data through 7 Oct 2026). Builds on 60 of its 76 references that are in the map. First posted on arXiv 28 Jun 2021.

Builds on

60 of 76 references.

References · 60

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2017Mask R-CNNcited by 7,259
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  10. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  11. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  12. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
38 more
  1. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  2. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  3. 2016Pyramid Scene Parsing Networkcited by 2,472
  4. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  5. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  6. 2017Deformable Convolutional Networkscited by 1,481
  7. 2015Spatial Transformer Networkscited by 1,423
  8. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  9. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  10. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  11. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  12. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  13. 2018Panoptic Segmentationcited by 796
  14. 2017Focal Loss for Dense Object Detectioncited by 749
  15. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  16. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  17. 2019Panoptic Feature Pyramid Networkscited by 644
  18. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
  19. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  20. 2020ResNeSt: Split-Attention Networkscited by 445
  21. 2019YOLACT: Real-time Instance Segmentationcited by 441
  22. 2018Rethinking ImageNet Pre-trainingcited by 376
  23. 2019Panoptic-DeepLabcited by 362
  24. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  25. 2016Dynamic Filter Networkscited by 318
  26. 2020Conditional Convolutions for Instance Segmentationcited by 314
  27. 2018Context Encoding for Semantic Segmentationcited by 289
  28. 2019SOLO: Segmenting Objects by Locationscited by 289
  29. 2019Mask Scoring R-CNNcited by 265
  30. 2016Associative Embedding: End-to-End Learning for Joint Detection and Groupingcited by 211
  31. 2019PolarMask: Single Shot Instance Segmentation with Polar Representationcited by 188
  32. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  33. 2019Expectation-Maximization Attention Networks for Semantic Segmentationcited by 147
  34. 2015End-to-end people detection in crowded scenescited by 140
  35. 2020Disentangled Non-Local Neural Networkscited by 132
  36. 2016Fully Convolutional Instance-aware Semantic Segmentationcited by 122
  37. 2015Instance-aware Semantic Segmentation via Multi-task Network Cascadescited by 120
  38. 2020Deep Snake for Real-Time Instance Segmentationcited by 111

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 42; 2023: 97; 2024: 34; 2025: 22; 2026: 9.

211 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,158
  2. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  3. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  4. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  5. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  6. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
  7. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 286
  8. 2021Decoupling Zero-Shot Semantic Segmentationcited by 198
  9. 2022RTMDet: An Empirical Study of Designing Real-Time Object Detectorscited by 192
  10. 2022Focal Modulation Networkscited by 186
  11. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  12. 2022DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detectioncited by 176
38 more
  1. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  2. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  3. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  4. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 126
  5. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  6. 2022Panoptic Scene Graph Generationcited by 114
  7. 2021Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformerscited by 109
  8. 2022SegViT: Semantic Segmentation with Plain Vision Transformerscited by 105
  9. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  10. 2023OneFormer3D: One Transformer for Unified Point Cloud Segmentationcited by 95
  11. 2023FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentationcited by 94
  12. 2022OpenEarthMap: A Benchmark Dataset for Global High-Resolution Land Cover Mappingcited by 80
  13. 2022Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentationcited by 79
  14. 2021Open-World Entity Segmentationcited by 74
  15. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 73
  16. 2021Mask Transfiner for High-Quality Instance Segmentationcited by 64
  17. 2023A Comprehensive Survey on Segment Anything Model for Vision and Beyondcited by 64
  18. 2022Sparse Instance Activation for Real-Time Instance Segmentationcited by 63
  19. 2022CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentationcited by 60
  20. 2022Temporally Efficient Vision Transformer for Video Instance Segmentationcited by 55
  21. 2023You Only Segment Once: Towards Real-Time Panoptic Segmentationcited by 40
  22. 2023See, Say, and Segment: Teaching LMMs to Overcome False Premisescited by 40
  23. 2023Lesion-aware Dynamic Kernel for Polyp Segmentationcited by 39
  24. 2023MP-Former: Mask-Piloted Transformer for Image Segmentationcited by 39
  25. 2023Open-vocabulary Panoptic Segmentation with Embedding Modulationcited by 39
  26. 2022Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentationcited by 38
  27. 2021PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentationcited by 36
  28. 20223D Instances as 1D Kernelscited by 36
  29. 2022TubeFormer-DeepLab: Video Mask Transformercited by 35
  30. 2023CLUSTSEG: Clustering for Universal Segmentationcited by 32
  31. 2023Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentationcited by 32
  32. 2022BoxTeacher: Exploring High-Quality Pseudo Labels for Weakly Supervised Instance Segmentationcited by 31
  33. 2023PhenoBench -- A Large Dataset and Benchmarks for Semantic Image Interpretation in the Agricultural Domaincited by 31
  34. 2022StructToken : Rethinking Semantic Segmentation with Structural Priorcited by 30
  35. 2023DaTaSeg: Taming a Universal Multi-Dataset Multi-Task Segmentation Modelcited by 30
  36. 2025X-SAM: From Segment Anything to Any Segmentationcited by 30
  37. 2024SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decompositioncited by 29
  38. 2023Betrayed by Captions: Joint Caption Grounding and Generation for Open Vocabulary Instance Segmentationcited by 28

50 most cited of 211 All 211 (JSON)