mlsanityData to 14 Oct 2026

CS-Mixer: A Cross-Scale Vision MLP Model with Spatial-Channel Mixing

arXiv 2308.13363 ↗First posted 25 Aug 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Despite their simpler information fusion designs compared with Vision Transformers and Convolutional Neural Networks, Vision MLP architectures have demonstrated strong performance and high data efficiency in recent research. However, existing works such as CycleMLP and Vision Permutator typically model spatial information in equal-size spatial regions and do not consider cross-scale spatial interactions. Further, their token mixers only model 1- or 2-axis correlations, avoiding 3-axis spatial-channel mixing due to its computational demands. We therefore propose CS-Mixer, a hierarchical Vision MLP that learns dynamic low-rank transformations for spatial-channel mixing through cross-scale local and global aggregation. The proposed methodology achieves competitive results on popular image recognition benchmarks without incurring substantially more compute. Our largest model, CS-Mixer-L, reaches 83.2% top-1 accuracy on ImageNet-1k with 13.7 GFLOPs and 94 M parameters.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

30 of 39 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  8. 2022A ConvNet for the 2020scited by 3,579
  9. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  10. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  11. 2017Random Erasing Data Augmentationcited by 1,239
  12. 2016Deep Networks with Stochastic Depthcited by 979
18 more
  1. 2021Going deeper with Image Transformerscited by 764
  2. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  3. 2021Pay Attention to MLPscited by 368
  4. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 360
  5. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 339
  6. 2021DeepViT: Towards Deeper Vision Transformercited by 289
  7. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  8. 2021CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attentioncited by 139
  9. 2021CycleMLP: A MLP-like Architecture for Dense Predictioncited by 126
  10. 2021Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognitioncited by 103
  11. 2021AS-MLP: An Axial Shifted MLP Architecture for Visioncited by 101
  12. 2021Scalable Vision Transformers with Hierarchical Poolingcited by 84
  13. 2021S2-MLP: Spatial-Shift MLP Architecture for Visioncited by 83
  14. 2021Exponential Moving Average Normalization for Self-supervised and Semi-supervised Learningcited by 78
  15. 2021Hire-MLP: Vision MLP via Hierarchical Rearrangementcited by 66
  16. 2022Mixing and Shifting: Exploiting Global and Local Dependencies in Vision MLPscited by 6
  17. 2021RaftMLP: How Much Can Be Done Without Attention and with Less Spatial Locality?cited by 5
  18. 2022gSwin: Gated MLP Vision Model with Hierarchical Structure of Shifted Windowcited by 5

Built on by

Not cited yet.