mlsanityData to 7 Oct 2026

CLUSTSEG: Clustering for Universal Segmentation

arXiv 2305.02187 ↗First posted 3 May 2023cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

We present CLUSTSEG, a general, transformer-based framework that tackles different image segmentation tasks (i.e., superpixel, semantic, instance, and panoptic) through a unified neural clustering scheme. Regarding queries as cluster centers, CLUSTSEG is innovative in two aspects:1) cluster centers are initialized in heterogeneous ways so as to pointedly address task-specific demands (e.g., instance- or category-level distinctiveness), yet without modifying the architecture; and 2) pixel-cluster assignment, formalized in a cross-attention fashion, is alternated with cluster center update, yet without learning additional parameters. These innovations closely link CLUSTSEG to EM clustering and make it a transparent and powerful framework that yields superior results across the above segmentation tasks.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 72 of its 92 references that are in the map. First posted on arXiv 3 May 2023.

Builds on

72 of 92 references.

References · 72

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017Mask R-CNNcited by 7,259
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  9. 2017Squeeze-and-Excitation Networkscited by 4,763
  10. 2022A ConvNet for the 2020scited by 3,545
  11. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
38 more
  1. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  2. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  3. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  4. 2017Non-local Neural Networkscited by 2,290
  5. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  6. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  7. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  8. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  9. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  10. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  11. 2018Panoptic Segmentationcited by 796
  12. 2019Panoptic Feature Pyramid Networkscited by 644
  13. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  14. 2019PointRend: Image Segmentation as Renderingcited by 439
  15. 2019Cascade R-CNN: High Quality Object Detection and Instance Segmentationcited by 428
  16. 2014Deeply-Supervised Netscited by 364
  17. 2019Panoptic-DeepLabcited by 362
  18. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  19. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  20. 2019SOLO: Segmenting Objects by Locationscited by 289
  21. 2019Mask Scoring R-CNNcited by 265
  22. 2021Exploring Cross-Image Pixel Contrast for Semantic Segmentationcited by 242
  23. 2015Conditional Random Fields as Recurrent Neural Networkscited by 213
  24. 2021K-Net: Towards Unified Image Segmentationcited by 211
  25. 2019UPSNet: A Unified Panoptic Segmentation Networkcited by 175
  26. 2021Instances as Queriescited by 157
  27. 2020BlendMask: Top-Down Meets Bottom-Up for Instance Segmentationcited by 144
  28. 2021Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformerscited by 109
  29. 2019TensorMask: A Foundation for Dense Object Segmentationcited by 101
  30. 2020Fully Convolutional Networks for Panoptic Segmentationcited by 92
  31. 2021Multi-Scale High-Resolution Vision Transformer for Semantic Segmentationcited by 92
  32. 2019Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidthcited by 88
  33. 2021SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentationcited by 87
  34. 2022GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Modelscited by 87
  35. 2018Superpixel Sampling Networkscited by 79
  36. 2016Deep Watershed Transform for Instance Segmentationcited by 77
  37. 2020Superpixel Segmentation with Fully Convolutional Networkscited by 72
  38. 2018Attention-guided Unified Network for Panoptic Segmentationcited by 71

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 10; 2025: 8; 2026: 3.

32 papers.

Sort:

By area:

  1. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 199
  2. 2023E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuningcited by 70
  3. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  4. 2023Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentationcited by 29
  5. 2024Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentationcited by 25
  6. 2023Clustering based Point Cloud Representation Learning for 3D Analysiscited by 23
  7. 2023DFormer: Diffusion-guided Transformer for Universal Image Segmentationcited by 19
  8. 2023LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoningcited by 17
  9. 2023PolyMaX: General Dense Prediction with Mask Transformercited by 14
  10. 2025Learning Clustering-based Prototypes for Compositional Zero-shot Learningcited by 13
  11. 2024Neural Clustering based Visual Representation Learningcited by 12
  12. 2024Clustering Propagation for Universal Medical Image Segmentationcited by 11
20 more
  1. 2024DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representationcited by 9
  2. 2023Learning Dynamic Query Combinations for Transformer-based Object Detection and Segmentationcited by 6
  3. 2023ClusterFormer: Clustering As A Universal Visual Learnercited by 6
  4. 2024Clustering for Protein Representation Learningcited by 6
  5. 2024General and Task-Oriented Video Segmentationcited by 6
  6. 2023Improving Scene Graph Generation with Superpixel-Based Interaction Learningcited by 4
  7. 2024Prototypical Transformer as Unified Motion Learnerscited by 4
  8. 2024BadPart: Unified Black-box Adversarial Patch Attacks against Pixel-wise Regression Taskscited by 3
  9. 2024SCASeg: Strip Cross-Attention for Efficient Semantic Segmentationcited by 2
  10. 2025Efficient Redundancy Reduction for Open-Vocabulary Semantic Segmentationcited by 2
  11. 2023Holistic Prototype Attention Network for Few-Shot VOScited by 1
  12. 2025Keep It Light! Simplifying Image Clustering Via Text-Free Adapterscited by 1
  13. 2025A Lightweight Convolution and Vision Transformer integrated model with Multi-scale Self-attention Mechanismcited by 1
  14. 2025ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generationcited by 1
  15. 2026SPEGC: Continual Test-Time Adaptation via Semantic-Prompt-Enhanced Graph Clustering for Medical Image Segmentationcited by 1
  16. 2025ChatStitch: Visualizing Through Structures via Surround-View Unsupervised Deep Image Stitching with Collaborative LLM-Agentscited by 0
  17. 2025CAM-Seg: A Continuous-valued Embedding Approach for Semantic Image Generationcited by 0
  18. 2025Intrinsic Dimensionality as a Model-Free Measure of Class Imbalancecited by 0
  19. 2026Computer Vision with a Superpixelation Cameracited by 0
  20. 2026Hyperbolic Hierarchical Clustering for Visual Representation Learningcited by 0