mlsanityData to 7 Oct 2026

MLP-Mixer: An all-MLP Architecture for Vision

arXiv 2105.01601 ↗First posted 4 May 2021cs.CV cs.AI cs.LGcited by 1,441

Not on the map. Why?

Cited by 28 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Convolutional Neural Networks (CNNs) are the go-to model for computer vision. Recently, attention-based networks, such as the Vision Transformer, have also become popular. In this paper we show that while convolutions and attention are both sufficient for good performance, neither of them are necessary. We present MLP-Mixer, an architecture based exclusively on multi-layer perceptrons (MLPs). MLP-Mixer contains two types of layers: one with MLPs applied independently to image patches (i.e. "mixing" the per-location features), and one with MLPs applied across patches (i.e. "mixing" spatial information). When trained on large datasets, or with modern regularization schemes, MLP-Mixer attains competitive scores on image classification benchmarks, with pre-training and inference cost comparable to state-of-the-art models. We hope that these results spark further research beyond the realms of well established CNNs and Transformers.

Key facts. Cited by 1,441 papers in the map (data through 7 Oct 2026). Builds on 47 of its 65 references that are in the map. First posted on arXiv 4 May 2021.

Builds on

47 of 65 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2016Layer Normalizationcited by 5,579
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  9. 2017Squeeze-and-Excitation Networkscited by 4,763
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2014Going Deeper with Convolutionscited by 4,231
  12. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
35 more
  1. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  2. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  3. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  4. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  5. 2017Non-local Neural Networkscited by 2,290
  6. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  7. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  8. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  9. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  10. 2016Deep Networks with Stochastic Depthcited by 973
  11. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  12. 2018Image Transformercited by 770
  13. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  14. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  15. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  16. 2013Network In Networkcited by 600
  17. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 549
  18. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  19. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  20. 2020Meta Pseudo Labelscited by 372
  21. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  22. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 359
  23. 2020Are we done with ImageNet?cited by 329
  24. 2019On the Relationship between Self-Attention and Convolutional Layerscited by 299
  25. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  26. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  27. 2019Pay Less Attention with Lightweight and Dynamic Convolutionscited by 211
  28. 2019Fixing the train-test resolution discrepancycited by 209
  29. 2020Synthesizer: Rethinking Self-Attention in Transformer Modelscited by 204
  30. 2021Revisiting ResNets: Improved Training and Scaling Strategiescited by 158
  31. 2021LambdaNetworks: Modeling Long-Range Interactions Without Attentioncited by 107
  32. 2021Involution: Inverting the Inherence of Convolution for Visual Recognitioncited by 99
  33. 2016Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Unitscited by 86
  34. 2020Towards Learning Convolutions from Scratchcited by 46
  35. 2019Non-discriminative data or weak model? On the relative importance of data and model resolutioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 145; 2022: 308; 2023: 308; 2024: 318; 2025: 217; 2026: 145.

1,441 papers.

Sort:

By area:

  1. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  3. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  4. 2024VMamba: Visual State Space Modelcited by 955
  5. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
  6. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  7. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  8. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  9. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 714
  10. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  11. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 572
  12. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
38 more
  1. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  2. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  3. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  4. 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
  5. 2022DeiT III: Revenge of the ViTcited by 417
  6. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  7. 2021Pay Attention to MLPscited by 367
  8. 2021ResNet strikes back: An improved training procedure in timmcited by 365
  9. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
  10. 2021FNet: Mixing Tokens with Fourier Transformscited by 343
  11. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  12. 2021XCiT: Cross-Covariance Image Transformerscited by 335
  13. 2021Choose a Transformer: Fourier or Galerkincited by 333
  14. 2024TimeMixer: Decomposable Multiscale Mixing for Time Series Forecastingcited by 333
  15. 2022How Do Vision Transformers Work?cited by 305
  16. 2022MAXIM: Multi-Axis MLP for Image Processingcited by 301
  17. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 292
  18. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  19. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  20. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  21. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  22. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 272
  23. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 259
  24. 2021Global Filter Networks for Image Classificationcited by 257
  25. 2021Knowledge distillation: A good teacher is patient and consistentcited by 255
  26. 2022EfficientFormer: Vision Transformers at MobileNet Speedcited by 240
  27. 2021Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformerscited by 237
  28. 2021Mobile-Former: Bridging MobileNet and Transformercited by 234
  29. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  30. 2023TSMixer: An All-MLP Architecture for Time Series Forecastingcited by 232
  31. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  32. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
  33. 2022Visual Attention Networkcited by 210
  34. 2022Patches Are All You Need?cited by 201
  35. 2025Real-Time Execution of Action Chunking Flow Policiescited by 199
  36. 2024Deep Time Series Models: A Comprehensive Survey and Benchmarkcited by 191
  37. 2021Are Transformers More Robust Than CNNs?cited by 187
  38. 2022Focal Modulation Networkscited by 186

50 most cited of 1,441 All 1,441 (JSON)