mlsanityData to 14 Oct 2026

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers

arXiv 2105.15203 ↗First posted 31 May 2021cs.CV cs.LGcited by 2,940

Not on the map. Why?

Cited by 139 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present SegFormer, a simple, efficient yet powerful semantic segmentation framework which unifies Transformers with lightweight multilayer perception (MLP) decoders. SegFormer has two appealing features: 1) SegFormer comprises a novel hierarchically structured Transformer encoder which outputs multiscale features. It does not need positional encoding, thereby avoiding the interpolation of positional codes which leads to decreased performance when the testing resolution differs from training. 2) SegFormer avoids complex decoders. The proposed MLP decoder aggregates information from different layers, and thus combining both local attention and global attention to render powerful representations. We show that this simple and lightweight design is the key to efficient segmentation on Transformers. We scale our approach up to obtain a series of models from SegFormer-B0 to SegFormer-B5, reaching significantly better performance and efficiency than previous counterparts. For example, SegFormer-B4 achieves 50.3% mIoU on ADE20K with 64M parameters, being 5x smaller and 2.2% better than the previous best method. Our best model, SegFormer-B5, achieves 84.0% mIoU on Cityscapes validation set and shows excellent zero-shot robustness on Cityscapes-C. Code will be released at: github.com/NVlabs/SegFormer.

Key facts. Cited by 2,940 papers in the map (data through 14 Oct 2026). Builds on 67 of its 87 references that are in the map. First posted on arXiv 31 May 2021.

Builds on

67 of 87 references.

References · 67

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2020End-to-End Object Detection with Transformerscited by 7,414
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  10. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  11. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  12. 2016Pyramid Scene Parsing Networkcited by 2,479
38 more
  1. 2017Non-local Neural Networkscited by 2,293
  2. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  3. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  4. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,133
  5. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  6. 2018Dual Attention Network for Scene Segmentationcited by 1,003
  7. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  8. 2020Pre-Trained Image Processing Transformercited by 893
  9. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  10. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  11. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 718
  12. 2021Transformer in Transformercited by 633
  13. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 576
  14. 2014Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFscited by 554
  15. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 553
  16. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 518
  17. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  18. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 465
  19. 2020ResNeSt: Split-Attention Networkscited by 445
  20. 2021TransReID: Transformer-based Object Re-Identificationcited by 435
  21. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  22. 2019Selective Kernel Networkscited by 403
  23. 2020TransTrack: Multiple Object Tracking with Transformercited by 371
  24. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  25. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  26. 2018Context Encoding for Semantic Segmentationcited by 290
  27. 2017ICNet for Real-Time Semantic Segmentation on High-Resolution Imagescited by 261
  28. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 260
  29. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  30. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
  31. 2017Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Networkcited by 208
  32. 2020How Much Position Information Do Convolutional Neural Networks Encode?cited by 172
  33. 2019Gated-SCNN: Gated Shape CNNs for Semantic Segmentationcited by 166
  34. 2018Pyramid Attention Network for Semantic Segmentationcited by 149
  35. 2019Expectation-Maximization Attention Networks for Semantic Segmentationcited by 147
  36. 2018OCNet: Object Context Network for Scene Parsingcited by 140
  37. 2020Context Prior for Scene Segmentationcited by 121
  38. 2018Learning a Discriminative Feature Network for Semantic Segmentationcited by 114

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 75; 2022: 366; 2023: 630; 2024: 691; 2025: 611; 2026: 567.

2,940 papers.

Sort:

By area:

  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  2. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  3. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  4. 2024Depth Anything V2cited by 1,422
  5. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  6. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  7. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  8. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  9. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  10. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  11. 2023Segment Everything Everywhere All at Oncecited by 548
  12. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 531
38 more
  1. 2022Continual Test-Time Domain Adaptationcited by 525
  2. 2021Image Segmentation Using Text and Image Promptscited by 512
  3. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  4. 2023Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splattingcited by 476
  5. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  6. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 363
  7. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
  8. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  9. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
  10. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  11. 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
  12. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 311
  13. 2022MAXIM: Multi-Axis MLP for Image Processingcited by 302
  14. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 300
  15. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 299
  16. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  17. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
  18. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 288
  19. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 285
  20. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 280
  21. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 264
  22. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 255
  23. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  24. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  25. 2023Personalize Segment Anything Model with One Shotcited by 250
  26. 2021Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformerscited by 248
  27. 2021TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditionscited by 244
  28. 2022A Transformer-Based Siamese Network for Change Detectioncited by 244
  29. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 241
  30. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  31. 2022CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformerscited by 231
  32. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  33. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  34. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  35. 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 218
  36. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  37. 2022Visual Attention Networkcited by 210
  38. 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 209

50 most cited of 2,940 All 2,940 (JSON)