mlsanityData to 14 Oct 2026

gSwin: Gated MLP Vision Model with Hierarchical Structure of Shifted Window

arXiv 2208.11718 ↗First posted 24 Aug 2022cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Following the success in language domain, the self-attention mechanism (transformer) is adopted in the vision domain and achieving great success recently. Additionally, as another stream, multi-layer perceptron (MLP) is also explored in the vision domain. These architectures, other than traditional CNNs, have been attracting attention recently, and many methods have been proposed. As one that combines parameter efficiency and performance with locality and hierarchy in image recognition, we propose gSwin, which merges the two streams; Swin Transformer and (multi-head) gMLP. We showed that our gSwin can achieve better accuracy on three vision tasks, image classification, object detection and semantic segmentation, than Swin Transformer, with smaller model size.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 55 of its 66 references that are in the map. First posted on arXiv 24 Aug 2022.

Builds on

55 of 66 references.

References · 55

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2020End-to-End Object Detection with Transformerscited by 7,414
  10. 2017Mask R-CNNcited by 7,277
  11. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  12. 2016Layer Normalizationcited by 5,669
38 more
  1. 2017Squeeze-and-Excitation Networkscited by 4,789
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  3. 2014Going Deeper with Convolutionscited by 4,241
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  5. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  6. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  7. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  9. 2021ViViT: A Video Vision Transformercited by 1,620
  10. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  11. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  12. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  13. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  14. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  15. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  16. 2021Video Swin Transformercited by 1,091
  17. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  18. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  19. 2021Multiscale Vision Transformerscited by 897
  20. 2018Image Transformercited by 772
  21. 2021Going deeper with Image Transformerscited by 764
  22. 2020PCT: Point cloud transformercited by 732
  23. 2021Transformer in Transformercited by 633
  24. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  25. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  26. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  27. 2020ResNeSt: Split-Attention Networkscited by 445
  28. 2021Pay Attention to MLPscited by 368
  29. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  30. 2014Recurrent Models of Visual Attentioncited by 349
  31. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 339
  32. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 334
  33. 2019On the Relationship between Self-Attention and Convolutional Layerscited by 300
  34. 2021DeepViT: Towards Deeper Vision Transformercited by 289
  35. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  36. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  37. 2016FractalNet: Ultra-Deep Neural Networks without Residualscited by 216
  38. 2021Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encodingcited by 207

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2025Hadamard product in deep learning: Introduction, Advances and Challengescited by 9
  2. 2024Discrete Cosine Transform Based Decorrelated Attention for Vision Transformerscited by 2
  3. 2024SpiralMLP: A Lightweight Vision MLP Architecturecited by 1
  4. 2023CS-Mixer: A Cross-Scale Vision MLP Model with Spatial-Channel Mixingcited by 0
  5. 2023SiT-MLP: A Simple MLP with Point-wise Topology Feature Learning for Skeleton-based Action Recognitioncited by 0