mlsanityData to 7 Oct 2026

Fully Transformer Networks for Semantic Image Segmentation

arXiv 2106.04108 ↗First posted 8 Jun 2021cs.CVcited by 24

Not on the map. Why?

Abstract (arXiv)

Transformers have shown impressive performance in various natural language processing and computer vision tasks, due to the capability of modeling long-range dependencies. Recent progress has demonstrated that combining such Transformers with CNN-based semantic image segmentation models is very promising. However, it is not well studied yet on how well a pure Transformer based approach can achieve for image segmentation. In this work, we explore a novel framework for semantic image segmentation, which is encoder-decoder based Fully Transformer Networks (FTN). Specifically, we first propose a Pyramid Group Transformer (PGT) as the encoder for progressively learning hierarchical features, meanwhile reducing the computation complexity of the standard Visual Transformer (ViT). Then, we propose a Feature Pyramid Transformer (FPT) to fuse semantic-level and spatial-level information from multiple levels of the PGT encoder for semantic image segmentation. Surprisingly, this simple baseline can achieve better results on multiple challenging semantic segmentation and face parsing benchmarks, including PASCAL Context, ADE20K, COCOStuff, and CelebAMask-HQ. The source code will be released on https://github.com/BR-IDL/PaddleViT.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 66 of its 86 references that are in the map. First posted on arXiv 8 Jun 2021.

Builds on

66 of 86 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2016Layer Normalizationcited by 5,579
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  9. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  10. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  12. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
38 more
  1. 2014Going Deeper with Convolutionscited by 4,231
  2. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  3. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  4. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  5. 2016Pyramid Scene Parsing Networkcited by 2,472
  6. 2017Non-local Neural Networkscited by 2,290
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  9. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  10. 2021Vision Transformers for Dense Predictioncited by 1,789
  11. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  12. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  13. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  14. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  15. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  16. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  17. 2016Deep Networks with Stochastic Depthcited by 973
  18. 2020Designing Network Design Spacescited by 954
  19. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  20. 2021Multiscale Vision Transformerscited by 896
  21. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  22. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  23. 2019Panoptic Feature Pyramid Networkscited by 644
  24. 2021Transformer in Transformercited by 632
  25. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  26. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
  27. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 549
  28. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  29. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  30. 2019Attention Augmented Convolutional Networkscited by 363
  31. 2018Context Encoding for Semantic Segmentationcited by 289
  32. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  33. 2021Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encodingcited by 207
  34. 2019Asymmetric Non-local Neural Networks for Semantic Segmentationcited by 196
  35. 2021Boundary IoU: Improving Object-Centric Image Segmentation Evaluationcited by 186
  36. 2018Visual Interpretability for Deep Learning: a Surveycited by 155
  37. 2020Context Prior for Scene Segmentationcited by 121
  38. 2017Learned in Translation: Contextualized Word Vectorscited by 104

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 10; 2023: 5; 2024: 1; 2025: 4.

24 papers.

Sort:

  1. 2022BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigscited by 123
  2. 2022Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentationcited by 114
  3. 2021Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attentioncited by 36
  4. 2021Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real Worldcited by 30
  5. 2022StructToken : Rethinking Semantic Segmentation with Structural Priorcited by 30
  6. 2023Semantic Diffusion Network for Semantic Segmentationcited by 30
  7. 2024Unified Language-driven Zero-shot Domain Adaptationcited by 30
  8. 2021Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistancecited by 22
  9. 2022Recent Advances in Vision Transformer: A Survey and Outlook of Recent Workcited by 21
  10. 2022Transformer Scale Gate for Semantic Segmentationcited by 17
  11. 2022Fully Transformer Network for Change Detection of Remote Sensing Imagescited by 17
  12. 2022GLPanoDepth: Global-to-Local Panoramic Depth Estimationcited by 16
12 more
  1. 2023UniNeXt: Exploring A Unified Architecture for Vision Recognitioncited by 16
  2. 2023TransY-Net:Learning Fully Transformer Networks for Change Detection of Remote Sensing Imagescited by 13
  3. 2023PRSeg: A Lightweight Patch Rotate MLP Decoder for Semantic Segmentationcited by 4
  4. 2022Feature Selective Transformer for Semantic Image Segmentationcited by 1
  5. 2023An intelligent modular real-time vision-based system for environment perceptioncited by 1
  6. 2021Reiterative Domain Aware Multi-Target Adaptationcited by 0
  7. 2022Borrowing from yourself: Faster future video segmentation with partial channel updatecited by 0
  8. 2022Fingerprint Pore Detection: A Surveycited by 0
  9. 2025Deeply Dual Supervised learning for melanoma recognitioncited by 0
  10. 2025Edge Detection for Organ Boundaries via Top Down Refinement and SubPixel Upsamplingcited by 0
  11. 2025DualResolution Residual Architecture with Artifact Suppression for Melanocytic Lesion Segmentationcited by 0
  12. 2025VesselRW: Weakly Supervised Subcutaneous Vessel Segmentation via Learned Random Walk Propagationcited by 0