mlsanityData to 7 Oct 2026

RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer

arXiv 2210.07124 ↗First posted 13 Oct 2022cs.CVcited by 31

Not on the map. Why?

Abstract (arXiv)

Recently, transformer-based networks have shown impressive results in semantic segmentation. Yet for real-time semantic segmentation, pure CNN-based approaches still dominate in this field, due to the time-consuming computation mechanism of transformer. We propose RTFormer, an efficient dual-resolution transformer for real-time semantic segmenation, which achieves better trade-off between performance and efficiency than CNN-based models. To achieve high inference efficiency on GPU-like devices, our RTFormer leverages GPU-Friendly Attention with linear complexity and discards the multi-head mechanism. Besides, we find that cross-resolution attention is more efficient to gather global context information for high-resolution branch by spreading the high level knowledge learned from low-resolution branch. Extensive experiments on mainstream benchmarks demonstrate the effectiveness of our proposed RTFormer, it achieves state-of-the-art on Cityscapes, CamVid and COCOStuff, and shows promising results on ADE20K. Code is available at PaddleSeg: https://github.com/PaddlePaddle/PaddleSeg.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 44 of its 58 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

44 of 58 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2017Squeeze-and-Excitation Networkscited by 4,763
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  10. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  11. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  12. 2016Pyramid Scene Parsing Networkcited by 2,472
32 more
  1. 2017Non-local Neural Networkscited by 2,290
  2. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  3. 2019Searching for MobileNetV3cited by 1,984
  4. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  5. 2021Vision Transformers for Dense Predictioncited by 1,789
  6. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  7. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  8. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  9. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  10. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  11. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  12. 2021Scaling Vision Transformerscited by 993
  13. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
  14. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  15. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  16. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 573
  17. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  18. 2017Relation Networks for Object Detectioncited by 305
  19. 2020BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentationcited by 294
  20. 2017ICNet for Real-Time Semantic Segmentation on High-Resolution Imagescited by 261
  21. 2018OCNet: Object Context Network for Scene Parsingcited by 140
  22. 2020Semantic Flow for Fast and Accurate Scene Parsingcited by 128
  23. 2021Rethinking BiSeNet For Real-time Semantic Segmentationcited by 124
  24. 2021Beyond Self-attention: External Attention using Two Linear Layers for Visual Taskscited by 109
  25. 2019DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentationcited by 100
  26. 2021Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenescited by 89
  27. 2019HarDNet: A Low Memory Traffic Networkcited by 76
  28. 2019FasterSeg: Searching for Faster Real-time Semantic Segmentationcited by 73
  29. 2019Customizable Architecture Search for Semantic Segmentationcited by 41
  30. 2019Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Searchcited by 39
  31. 2021PaddleSeg: A High-Efficient Development Toolkit for Image Segmentationcited by 33
  32. 2019Graph-guided Architecture Search for Real-time Semantic Segmentationcited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 14; 2025: 6; 2026: 2.

31 papers.

Sort:

By area:

  1. 2023Content-aware Token Sharing for Efficient Semantic Segmentation with Vision Transformerscited by 22
  2. 2023SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentationcited by 22
  3. 2024Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentationcited by 21
  4. 2025Benchmarking Multi-modal Semantic Segmentation under Sensor Failures: Missing and Noisy Modality Robustnesscited by 18
  5. 2022Mask-FPAN: Semi-Supervised Face Parsing in the Wild With De-Occlusion and UV GANcited by 16
  6. 2024Embedding-Free Transformer with Inference Spatial Reduction for Efficient Semantic Segmentationcited by 14
  7. 2024MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selectioncited by 12
  8. 2025MemorySAM: Memorize Modalities and Semantics with Segment Anything Model 2 for Multi-modal Semantic Segmentationcited by 9
  9. 2024SituationAdapt: Contextual UI Optimization in Mixed Reality with Situation Awareness via LLM Reasoningcited by 8
  10. 2024SERNet-Former: Semantic Segmentation by Efficient Residual Network with Attention-Boosting Gates and Attention-Fusion Networkscited by 7
  11. 2023PP-MobileSeg: Explore the Fast and Accurate Semantic Segmentation Model on Mobile Devicescited by 6
  12. 2023Fast-StrucTexT: An Efficient Hourglass Transformer with Modality-guided Dynamic Token Merge for Document Understandingcited by 6
19 more
  1. 2024Transformers and Language Models in Form Understanding: A Comprehensive Review of Scanned Document Analysiscited by 6
  2. 2024Make a Strong Teacher with Label Assistance: A Novel Knowledge Distillation Approach for Semantic Segmentationcited by 5
  3. 2024Communicate Less, Synthesize the Rest: Latency-aware Intent-based Generative Semantic Multicasting with Diffusion Modelscited by 5
  4. 2025Revisiting Efficient Semantic Segmentation: Learning Offsets for Better Spatial and Class Feature Alignmentcited by 5
  5. 2023Principles of Forgetting in Domain-Incremental Semantic Segmentation in Adverse Weather Conditionscited by 4
  6. 2023EFormer: Enhanced Transformer towards Semantic-Contour Features of Foreground for Portraits Mattingcited by 4
  7. 2024Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insightscited by 4
  8. 2024DSNet: A Novel Way to Use Atrous Convolutions in Semantic Segmentationcited by 4
  9. 2024AVESFormer: Efficient Transformer Design for Real-Time Audio-Visual Segmentationcited by 4
  10. 2023RT-K-Net: Revisiting K-Net for Real-Time Panoptic Segmentationcited by 1
  11. 2023Bilateral Network with Residual U-blocks and Dual-Guided Attention for Real-time Semantic Segmentationcited by 1
  12. 2024A feature refinement module for light-weight semantic segmentation networkcited by 1
  13. 2025HDBFormer: Efficient RGB-D Semantic Segmentation with A Heterogeneous Dual-Branch Frameworkcited by 1
  14. 2025TSLA: A Task-Specific Learning Adaptation for Semantic Segmentation on Autonomous Vehicles Platformcited by 1
  15. 2024A Decoding Scheme with Successive Aggregation of Multi-Level Features for Light-Weight Semantic Segmentationcited by 0
  16. 2024MGNiceNet: Unified Monocular Geometric Scene Understandingcited by 0
  17. 2025WaveSeg: Enhancing Segmentation Precision via High-Frequency Prior and Mamba-Driven Spectrum Decompositioncited by 0
  18. 2026Cross-Stage Attention Propagation for Efficient Semantic Segmentationcited by 0
  19. 2026Overcoming "Physics Shock" in Earth Observation A Heteroscedastic Uncertainty Framework for PINN-based Flood Inferencecited by 0