Data to 14 Oct 2026

Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

arXiv 2012.15840 ↗First posted 31 Dec 2020cs.CVcited by 1,369

Not on the map. Why?

Cited by 30 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Most recent semantic segmentation methods adopt a fully-convolutional network (FCN) with an encoder-decoder architecture. The encoder progressively reduces the spatial resolution and learns more abstract/semantic visual concepts with larger receptive fields. Since context modeling is critical for segmentation, the latest efforts have been focused on increasing the receptive field, through either dilated/atrous convolutions or inserting attention modules. However, the encoder-decoder based FCN architecture remains unchanged. In this paper, we aim to provide an alternative perspective by treating semantic segmentation as a sequence-to-sequence prediction task. Specifically, we deploy a pure transformer (ie, without convolution and resolution reduction) to encode an image as a sequence of patches. With the global context modeled in every layer of the transformer, this encoder can be combined with a simple decoder to provide a powerful segmentation model, termed SEgmentation TRansformer (SETR). Extensive experiments show that SETR achieves new state of the art on ADE20K (50.28% mIoU), Pascal Context (55.83% mIoU) and competitive results on Cityscapes. Particularly, we achieve the first position in the highly competitive ADE20K test server leaderboard on the day of submission.

Key facts. Cited by 1,369 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 220; 2022: 355; 2023: 361; 2024: 232; 2025: 138; 2026: 63.

1,369 papers.

Sort:

By area:

2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
2022Visual Prompt Tuningcited by 1,681
2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
2021ViViT: A Video Vision Transformercited by 1,620
2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
2021Transformers in Vision: A Surveycited by 1,013

38 more

2021UNETR: Transformers for 3D Medical Image Segmentationcited by 959
2023Vision Transformers Need Registerscited by 871
2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 866
2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
2021Segmenter: Transformer for Semantic Segmentationcited by 855
2023Vision-Language Models for Vision Tasks: A Surveycited by 839
2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
2021Transformer in Transformercited by 633
2021ImageNet-21K Pretraining for the Massescited by 613
2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
2022Vision Transformer Adapter for Dense Predictionscited by 565
2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 531
2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 527
2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 519
2021Image Segmentation Using Text and Image Promptscited by 512
2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
2021PointCLIP: Point Cloud Understanding by CLIPcited by 463
2021TransReID: Transformer-based Object Re-Identificationcited by 435
2021Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modelingcited by 431
2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 422
2021Conditional Positional Encodings for Vision Transformerscited by 406
2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
2021XCiT: Cross-Covariance Image Transformerscited by 338
2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
2021Medical Transformer: Gated Axial-Attention for Medical Image Segmentationcited by 320
2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 320
2021CMT: Convolutional Neural Networks Meet Vision Transformerscited by 310
2021Remote Sensing Image Change Detection with Transformerscited by 306
2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 300
2023OccFormer: Dual-path Transformer for Vision-based 3D Semantic Occupancy Predictioncited by 293
2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
2021DeepViT: Towards Deeper Vision Transformercited by 289

50 most cited of 1,369 All 1,369 (JSON)