mlsanityData to 14 Oct 2026

MSG-Transformer: Exchanging Local Spatial Information by Manipulating Messenger Tokens

arXiv 2105.15168 ↗First posted 31 May 2021cs.CV cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

Transformers have offered a new methodology of designing neural networks for visual recognition. Compared to convolutional networks, Transformers enjoy the ability of referring to global features at each stage, yet the attention module brings higher computational overhead that obstructs the application of Transformers to process high-resolution visual data. This paper aims to alleviate the conflict between efficiency and flexibility, for which we propose a specialized token for each region that serves as a messenger (MSG). Hence, by manipulating these MSG tokens, one can flexibly exchange visual information across regions and the computational complexity is reduced. We then integrate the MSG token into a multi-scale architecture named MSG-Transformer. In standard image classification and object detection, MSG-Transformer achieves competitive performance and the inference on both GPU and CPU is accelerated. Code is available at https://github.com/hustvl/MSG-Transformer.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 53 of its 63 references that are in the map. First posted on arXiv 31 May 2021.

Builds on

53 of 63 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  8. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  10. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  11. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  12. 2020End-to-End Object Detection with Transformerscited by 7,414
38 more
  1. 2017Mask R-CNNcited by 7,277
  2. 2016Densely Connected Convolutional Networkscited by 6,720
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2014Going Deeper with Convolutionscited by 4,241
  9. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  10. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  11. 2015SSD: Single Shot MultiBox Detectorcited by 3,130
  12. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  13. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  14. 2017Non-local Neural Networkscited by 2,293
  15. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
  16. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  17. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  18. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,369
  19. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  20. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  21. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  22. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  23. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  24. 2020Designing Network Design Spacescited by 960
  25. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  26. 2021Going deeper with Image Transformerscited by 764
  27. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 718
  28. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  29. 2021Transformer in Transformercited by 633
  30. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  31. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  32. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  33. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  34. 2020Exploring Self-attention for Image Recognitioncited by 313
  35. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
  36. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  37. 2019Local Relation Networks for Image Recognitioncited by 210
  38. 2020Rethinking Positional Encoding in Language Pre-trainingcited by 197

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 14; 2023: 16; 2024: 7; 2025: 0; 2026: 2.

46 papers.

Sort:

By area:

  1. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  2. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  3. 2022HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentationcited by 95
  4. 2022CF-ViT: A General Coarse-to-Fine Method for Vision Transformercited by 63
  5. 2022Temporally Efficient Vision Transformer for Video Instance Segmentationcited by 55
  6. 2023Vision Transformer with Quadrangle Attentioncited by 46
  7. 2024A Survey on Transformer Compressioncited by 46
  8. 2022MixFormer: Mixing Features across Windows and Dimensionscited by 43
  9. 2022VSA: Learning Varied-Size Window Attention in Vision Transformerscited by 43
  10. 2023MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognitioncited by 43
  11. 2022LightViT: Towards Light-Weight Convolution-Free Vision Transformerscited by 40
  12. 2021Sparse MLP for Image Recognition: Is Self-Attention Really Necessary?cited by 38
34 more
  1. 2021Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attentioncited by 36
  2. 2022Towards Data-Efficient Detection Transformerscited by 34
  3. 2021Scaled ReLU Matters for Training Vision Transformerscited by 29
  4. 2022Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasetscited by 29
  5. 2023TransFace++: Rethinking the Face Recognition Paradigm with a Focus on Accuracy, Efficiency, and Securitycited by 22
  6. 2023SCTNet: Single-Branch CNN with Transformer Semantic Information for Real-Time Segmentationcited by 22
  7. 2023DAT++: Spatially Dynamic Vision Transformer with Deformable Attentioncited by 21
  8. 2023Robustifying Token Attention for Vision Transformerscited by 19
  9. 2022Dynamic Group Transformer: A General Vision Transformer Backbone with Dynamic Group Attentioncited by 14
  10. 2022Beyond Masking: Demystifying Token-Based Pre-Training for Vision Transformerscited by 14
  11. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  12. 2024Depth-Wise Convolutions in Vision Transformers for Efficient Training on Small Datasetscited by 14
  13. 2024A Lightweight Sparse Focus Transformer for Remote Sensing Image Change Captioningcited by 13
  14. 2024ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attentioncited by 13
  15. 2021Tracking Instances as Queriescited by 7
  16. 2022TransPillars: Coarse-to-Fine Aggregation for Multi-Frame 3D Object Detectioncited by 6
  17. 2022DQnet: Cross-Model Detail Querying for Camouflaged Object Detectioncited by 6
  18. 2023MobileInst: Video Instance Segmentation on the Mobilecited by 6
  19. 2023Multi-level Multiple Instance Learning with Transformer for Whole Slide Image Classificationcited by 6
  20. 2023Graph-Segmenter: Graph Transformer with Boundary-aware Attention for Semantic Segmentationcited by 6
  21. 2023ClusterFormer: Clustering As A Universal Visual Learnercited by 6
  22. 2023Factorization Vision Transformer: Modeling Long Range Dependency with Local Window Costcited by 5
  23. 2024Masked Spatial Propagation Network for Sparsity-Adaptive Depth Refinementcited by 5
  24. 2021What Makes for Hierarchical Vision Transformer?cited by 4
  25. 2021Global Interaction Modelling in Vision Transformer via Super Tokenscited by 4
  26. 2024Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insightscited by 4
  27. 2023CEDNet: A Cascade Encoder-Decoder Network for Dense Predictioncited by 3
  28. 2023Circuit as Set of Pointscited by 3
  29. 20223D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volumecited by 2
  30. 2022ViT-LSLA: Vision Transformer with Light Self-Limited-Attentioncited by 2
  31. 2023Versatile Depth Estimator Based on Common Relative Depth Estimation and Camera-Specific Relative-to-Metric Depth Conversioncited by 1
  32. 2023AxWin Transformer: A Context-Aware Vision Transformer Backbone with Axial Windowscited by 1
  33. 2026RAViT: Resolution-Adaptive Vision Transformercited by 0
  34. 2026Advancing Vision Transformer with Enhanced Spatial Priorscited by 0