mlsanityData to 14 Oct 2026

TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation

arXiv 2204.05525 ↗First posted 12 Apr 2022cs.CVcited by 82

Not on the map. Why?

Abstract (arXiv)

Although vision transformers (ViTs) have achieved great success in computer vision, the heavy computational cost hampers their applications to dense prediction tasks such as semantic segmentation on mobile devices. In this paper, we present a mobile-friendly architecture named Token Pyramid Vision Transformer (TopFormer). The proposed TopFormer takes Tokens from various scales as input to produce scale-aware semantic features, which are then injected into the corresponding tokens to augment the representation. Experimental results demonstrate that our method significantly outperforms CNN- and ViT-based networks across several semantic segmentation datasets and achieves a good trade-off between accuracy and latency. On the ADE20K dataset, TopFormer achieves 5% higher accuracy in mIoU than MobileNetV3 with lower latency on an ARM-based mobile device. Furthermore, the tiny version of TopFormer achieves real-time inference on an ARM-based mobile device with competitive results. The code and models are available at: https://github.com/hustvl/TopFormer

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). Builds on 53 of its 60 references that are in the map. First posted on arXiv 12 Apr 2022.

Builds on

53 of 60 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  11. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  12. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
38 more
  1. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  2. 2016Pyramid Scene Parsing Networkcited by 2,479
  3. 2017Non-local Neural Networkscited by 2,293
  4. 2019Searching for MobileNetV3cited by 1,994
  5. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  6. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  7. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  8. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  9. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  10. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  11. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  12. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  13. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  14. 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 864
  15. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  16. 2019Panoptic Feature Pyramid Networkscited by 645
  17. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
  18. 2019Segmentation Transformer: Object-Contextual Representations for Semantic Segmentationcited by 576
  19. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  20. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  21. 2019GhostNet: More Features from Cheap Operationscited by 410
  22. 2020Exploring Self-attention for Image Recognitioncited by 313
  23. 2017ICNet for Real-Time Semantic Segmentation on High-Resolution Imagescited by 261
  24. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  25. 2021Mobile-Former: Bridging MobileNet and Transformercited by 235
  26. 2019Local Relation Networks for Image Recognitioncited by 210
  27. 2021HRFormer: High-Resolution Transformer for Dense Predictioncited by 149
  28. 2021Patch Slimming for Efficient Vision Transformerscited by 133
  29. 2020Semantic Flow for Fast and Accurate Scene Parsingcited by 128
  30. 2021Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformercited by 115
  31. 2019In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Imagescited by 100
  32. 2019DFANet: Deep Feature Aggregation for Real-Time Semantic Segmentationcited by 100
  33. 2018ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Networkcited by 90
  34. 2019AdderNet: Do We Really Need Multiplications in Deep Learning?cited by 82
  35. 2019FasterSeg: Searching for Faster Real-time Semantic Segmentationcited by 73
  36. 2020Rethinking Bottleneck Structure for Efficient Mobile Network Designcited by 54
  37. 2020AlignSeg: Feature-Aligned Segmentation Networkscited by 53
  38. 2020Model Rubik's Cube: Twisting Resolution, Depth and Width for TinyNetscited by 49

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 31; 2024: 19; 2025: 14; 2026: 10.

82 papers.

Sort:

By area:

  1. 2024YOLOv10: Real-Time End-to-End Object Detectioncited by 515
  2. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 241
  3. 2022ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentationcited by 178
  4. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  5. 2023RepViT: Revisiting Mobile CNN From ViT Perspectivecited by 123
  6. 2022Rethinking Vision Transformers for MobileNet Size and Speedcited by 121
  7. 2022In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimationcited by 51
  8. 2023Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanismcited by 51
  9. 2023SpectFormer: Frequency and Attention is what you need in a Vision Transformercited by 48
  10. 2023EdgeSAM: Prompt-In-the-Loop Distillation for SAMcited by 41
  11. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  12. 2023Semantic Image Segmentation: Two Decades of Researchcited by 27
38 more
  1. 2024Context-Guided Spatial Feature Reconstruction for Efficient Semantic Segmentationcited by 25
  2. 2023DiffI2I: Efficient Diffusion Model for Image-to-Image Translationcited by 24
  3. 2023Content-aware Token Sharing for Efficient Semantic Segmentation with Vision Transformerscited by 22
  4. 2023SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentationcited by 22
  5. 2024Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentationcited by 22
  6. 2023SeaFormer++: Squeeze-enhanced Axial Transformer for Mobile Visual Recognitioncited by 18
  7. 2024Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Losscited by 18
  8. 2024Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Modelcited by 18
  9. 2023Scattering Vision Transformer: Spectral Mixing Matterscited by 17
  10. 2025LSNet: See Large, Focus Smallcited by 15
  11. 2023Vision Transformer Adapters for Generalizable Multitask Learningcited by 14
  12. 2024Embedding-Free Transformer with Inference Spatial Reduction for Efficient Semantic Segmentationcited by 14
  13. 2024ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attentioncited by 13
  14. 2022Multi-manifold Attention for Vision Transformerscited by 12
  15. 2023CoFiI2P: Coarse-to-Fine Correspondences for Image-to-Point Cloud Registrationcited by 12
  16. 2024MetaSeg: MetaFormer-based Global Contexts-aware Network for Efficient Semantic Segmentationcited by 12
  17. 2024MAGIC++: Efficient and Resilient Modality-Agnostic Semantic Segmentation via Hierarchical Modality Selectioncited by 12
  18. 2023Residual Spatial Fusion Network for RGB-Thermal Semantic Segmentationcited by 10
  19. 2023Isomer: Isomerous Transformer for Zero-shot Video Object Segmentationcited by 10
  20. 2022Self-Attentive Pooling for Efficient Deep Learningcited by 9
  21. 2023SBCFormer: Lightweight Network Capable of Full-size ImageNet Classification at 1 FPS on Single Board Computerscited by 9
  22. 2023Self-supervised Hypergraphs for Learning Multiple World Interpretationscited by 8
  23. 2024Fast-Convergent and Communication-Alleviated Heterogeneous Hierarchical Federated Learning in Autonomous Drivingcited by 8
  24. 2025Adaptive Frequency Enhancement Network for Remote Sensing Image Semantic Segmentationcited by 8
  25. 2025EDM: Efficient Deep Feature Matchingcited by 7
  26. 2023Consistency Regularisation in Varying Contexts and Feature Perturbations for Semi-Supervised Semantic Segmentation of Histology Imagescited by 6
  27. 2023Efficiency 360: Efficient Vision Transformerscited by 6
  28. 2023MobileInst: Video Instance Segmentation on the Mobilecited by 6
  29. 2023PP-MobileSeg: Explore the Fast and Accurate Semantic Segmentation Model on Mobile Devicescited by 6
  30. 2023Multi-Task Hypergraphs for Semi-supervised Learning using Earth Observationscited by 6
  31. 2024RMP-SAM: Towards Real-Time Multi-Purpose Segment Anythingcited by 6
  32. 2022DPNet: Dual-Path Network for Real-time Object Detection with Lightweight Attentioncited by 5
  33. 2023eWaSR -- an embedded-compute-ready maritime obstacle detection networkcited by 5
  34. 2023Mobile-Seed: Joint Semantic Segmentation and Boundary Detection for Mobile Robotscited by 5
  35. 2024HAFormer: Unleashing the Power of Hierarchy-Aware Features for Lightweight Semantic Segmentationcited by 5
  36. 2025FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Drivingcited by 5
  37. 2025Revisiting Efficient Semantic Segmentation: Learning Offsets for Better Spatial and Class Feature Alignmentcited by 5
  38. 2023M3Net: Multilevel, Mixed and Multistage Attention Network for Salient Object Detectioncited by 4

50 most cited of 82 All 82 (JSON)