mlsanityData to 14 Oct 2026

Segmenter: Transformer for Semantic Segmentation

arXiv 2105.05633 ↗First posted 12 May 2021cs.CV cs.AI cs.LGcited by 855

Not on the map. Why?

Cited by 27 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Image segmentation is often ambiguous at the level of individual image patches and requires contextual information to reach label consensus. In this paper we introduce Segmenter, a transformer model for semantic segmentation. In contrast to convolution-based methods, our approach allows to model global context already at the first layer and throughout the network. We build on the recent Vision Transformer (ViT) and extend it to semantic segmentation. To do so, we rely on the output embeddings corresponding to image patches and obtain class labels from these embeddings with a point-wise linear decoder or a mask transformer decoder. We leverage models pre-trained for image classification and show that we can fine-tune them on moderate sized datasets available for semantic segmentation. The linear decoder allows to obtain excellent results already, but the performance can be further improved by a mask transformer generating class masks. We conduct an extensive ablation study to show the impact of the different parameters, in particular the performance is better for large models and small patch sizes. Segmenter attains excellent results for semantic segmentation. It outperforms the state of the art on both ADE20K and Pascal Context datasets and is competitive on Cityscapes.

Key facts. Cited by 855 papers in the map (data through 14 Oct 2026). Builds on 48 of its 68 references that are in the map. First posted on arXiv 12 May 2021.

Builds on

48 of 68 references.

References · 48

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  8. 2016Layer Normalizationcited by 5,669
  9. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  10. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  12. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
36 more
  1. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  2. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  3. 2016Pyramid Scene Parsing Networkcited by 2,479
  4. 2017Non-local Neural Networkscited by 2,293
  5. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  6. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  7. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  8. 2021ViViT: A Video Vision Transformercited by 1,620
  9. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  10. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  11. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,133
  12. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  13. 2018Dual Attention Network for Scene Segmentationcited by 1,003
  14. 2016Deep Networks with Stochastic Depthcited by 979
  15. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 718
  16. 2020Image Segmentation Using Deep Learning: A Surveycited by 660
  17. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 576
  18. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 554
  19. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 465
  20. 2020ResNeSt: Split-Attention Networkscited by 445
  21. 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 431
  22. 2019Panoptic-DeepLabcited by 363
  23. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  24. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  25. 2019Asymmetric Non-local Neural Networks for Semantic Segmentationcited by 196
  26. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  27. 2020Multiscale Deep Equilibrium Modelscited by 151
  28. 2018OCNet: Object Context Network for Scene Parsingcited by 140
  29. 2020Disentangled Non-Local Neural Networkscited by 132
  30. 2020Context Prior for Scene Segmentationcited by 121
  31. 2018How to Start Training: The Effect of Initialization and Architecturecited by 109
  32. 2016Full-Resolution Residual Networks for Semantic Segmentation in Street Scenescited by 71
  33. 2019Adaptive Context Network for Scene Parsingcited by 51
  34. 2020Evolution of Image Segmentation using Deep Convolutional Neural Network: A Surveycited by 28
  35. 2017Deep Semantic Segmentation for Automated Driving: Taxonomy, Roadmap and Challengescited by 24
  36. 2018Virtual-to-Real: Learning to Control in Visual Semantic Segmentationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 42; 2022: 198; 2023: 260; 2024: 181; 2025: 117; 2026: 57.

855 papers.

Sort:

By area:

  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  2. 2022Visual Prompt Tuningcited by 1,681
  3. 2024Depth Anything V2cited by 1,422
  4. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  5. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  6. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  7. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  8. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  9. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  10. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  11. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 527
  12. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
38 more
  1. 2021Image Segmentation Using Text and Image Promptscited by 512
  2. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
  3. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  4. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  5. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  6. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
  7. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 288
  8. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 280
  9. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  10. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  11. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  12. 2022Transformers in Medical Imaging: A Surveycited by 245
  13. 2024LocalMamba: Visual State Space Model with Windowed Selective Scancited by 236
  14. 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 231
  15. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  16. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
  17. 2023Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIPcited by 201
  18. 2022Focal Modulation Networkscited by 186
  19. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 186
  20. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  21. 2023SCLIP: Rethinking Self-Attention for Dense Vision-Language Inferencecited by 170
  22. 2021UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagerycited by 167
  23. 2021VOLO: Vision Outlooker for Visual Recognitioncited by 165
  24. 2023Language-Driven Representation Learning for Roboticscited by 165
  25. 2022HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentationcited by 164
  26. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  27. 2022FlexiViT: One Model for All Patch Sizescited by 152
  28. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  29. 2022Rethinking Semantic Segmentation: A Prototype Viewcited by 140
  30. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  31. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 128
  32. 2022UIU-Net: U-Net in U-Net for Infrared Small Object Detectioncited by 127
  33. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  34. 2022CLIP is Also an Efficient Segmenter: A Text-Driven Approach for Weakly Supervised Semantic Segmentationcited by 118
  35. 2021A Survey of Visual Transformerscited by 117
  36. 2023Delivering Arbitrary-Modal Semantic Segmentationcited by 117
  37. 2024Generative Camera Dolly: Extreme Monocular Dynamic Novel View Synthesiscited by 117
  38. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 117

50 most cited of 855 All 855 (JSON)