mlsanityData to 7 Oct 2026

Transformer Scale Gate for Semantic Segmentation

arXiv 2205.07056 ↗First posted 14 May 2022cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Effectively encoding multi-scale contextual information is crucial for accurate semantic segmentation. Existing transformer-based segmentation models combine features across scales without any selection, where features on sub-optimal scales may degrade segmentation outcomes. Leveraging from the inherent properties of Vision Transformers, we propose a simple yet effective module, Transformer Scale Gate (TSG), to optimally combine multi-scale features.TSG exploits cues in self and cross attentions in Vision Transformers for the scale selection. TSG is a highly flexible plug-and-play module, and can easily be incorporated with any encoder-decoder-based hierarchical vision Transformer architecture. Extensive experiments on the Pascal Context and ADE20K datasets demonstrate that our feature selection strategy achieves consistent gains.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 14 May 2022.

Builds on

35 of 52 references.

References · 35

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  7. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  8. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  9. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  10. 2016Pyramid Scene Parsing Networkcited by 2,472
  11. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  12. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
23 more
  1. 2021Vision Transformers for Dense Predictioncited by 1,789
  2. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  3. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  4. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  5. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  6. 2021Transformers in Vision: A Surveycited by 1,009
  7. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  8. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  9. 2016RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentationcited by 463
  10. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  11. 2018TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wildcited by 342
  12. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 286
  13. 2015Attention to Scale: Scale-aware Semantic Image Segmentationcited by 169
  14. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  15. 2021MPViT: Multi-Path Vision Transformer for Dense Predictioncited by 122
  16. 2021Multi-Scale High-Resolution Vision Transformer for Semantic Segmentationcited by 92
  17. 2021P2T: Pyramid Pooling Transformer for Scene Understandingcited by 84
  18. 2019GFF: Gated Fully Fusion for Semantic Segmentationcited by 57
  19. 2016Semantic Object Parsing with Graph LSTMcited by 44
  20. 2022ProposalCLIP: Unsupervised Open-Category Object Proposal Generation via Exploiting CLIP Cuescited by 42
  21. 2022Lawin Transformer: Improving Semantic Segmentation Transformer with Multi-Scale Representations via Large Window Attentioncited by 30
  22. 2021Fully Transformer Networks for Semantic Image Segmentationcited by 24
  23. 2022Pyramid Fusion Transformer for Semantic Segmentationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 6; 2025: 5; 2026: 2.

17 papers.

Sort:

By area:

  1. 2024ALGM: Adaptive Local-then-Global Token Merging for Efficient Semantic Segmentation with Plain Vision Transformerscited by 28
  2. 2024Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Expertscited by 15
  3. 2023Open-Vocabulary Object Detection via Scene Graph Discoverycited by 7
  4. 2023Soft labelling for semantic segmentation: Bringing coherence to label down-samplingcited by 5
  5. 2023Unified Open-Vocabulary Dense Visual Predictioncited by 4
  6. 2023High-level Feature Guided Decoding for Semantic Segmentationcited by 2
  7. 2024FANet: Feature Amplification Network for Semantic Segmentation in Cluttered Backgroundcited by 2
  8. 2025Image Recognition with Online Lightweight Vision Transformer: A Surveycited by 2
  9. 2024Large coordinate kernel attention network for lightweight image super-resolutioncited by 1
  10. 2025VPNeXt -- Rethinking Dense Decoding for Plain Vision Transformercited by 1
  11. 2024Optimized Information Flow for Transformer Trackingcited by 0
  12. 2024Hybrid Deep Learning-Based for Enhanced Occlusion Segmentation in PICU Patient Monitoringcited by 0
5 more
  1. 2025Efficient Star Distillation Attention Network for Lightweight Image Super-Resolutioncited by 0
  2. 2025MPT: Motion Prompt Tuning for Micro-Expression Recognitioncited by 0
  3. 2025Classifier Enhancement Using Extended Context and Domain Experts for Semantic Segmentationcited by 0
  4. 2026Fluxamba: Topology-Aware Anisotropic State Space Models for Geological Lineament Segmentation in Multi-Source Remote Sensingcited by 0
  5. 2026Contextrast++: Robust Multi-Scale Contextual Contrastive Learning for Semantic Segmentationcited by 0