# Strip-MLP: Efficient Token Interaction for Vision MLP

arXiv: https://arxiv.org/abs/2307.11458 · HTML: https://mlsanity.com/paper/2307.11458/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 44 references that are in the map.
- First posted on arXiv 21 Jul 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Pay Attention to MLPs](https://mlsanity.com/paper/2105.08050/) (2021)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [CycleMLP: A MLP-like Architecture for Dense Prediction](https://mlsanity.com/paper/2107.10224/) (2021)
- [Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition](https://mlsanity.com/paper/2106.12368/) (2021)
- [Hire-MLP: Vision MLP via Hierarchical Rearrangement](https://mlsanity.com/paper/2108.13341/) (2021)
- [Are we ready for a new paradigm shift? A Survey on Visual Deep MLP](https://mlsanity.com/paper/2111.04060/) (2021)

## Built on by (most cited first, 12 of 12)

- [Full-resolution MLPs Empower Medical Dense Prediction](https://mlsanity.com/paper/2311.16707/) (2023)
- [QG-VTC: Question-Guided Visual Token Compression in MLLMs for Efficient VQA](https://mlsanity.com/paper/2504.00654/) (2025)
- [Open-Det: An Efficient Learning Framework for Open-Ended Detection](https://mlsanity.com/paper/2505.20639/) (2025)
- [FViT: A Focal Vision Transformer with Gabor Filter](https://mlsanity.com/paper/2402.11303/) (2024)
- [Team up GBDTs and DNNs: Advancing Efficient and Effective Tabular Prediction with Tree-hybrid MLPs](https://mlsanity.com/paper/2407.09790/) (2024)
- [Cross-DINO: Cross the Deep MLP and Transformer for Small Object Detection](https://mlsanity.com/paper/2505.21868/) (2025)
- [D2-MLP: Dynamic Decomposed MLP Mixer for Medical Image Segmentation](https://mlsanity.com/paper/2409.08905/) (2024)
- [EffOWT: Transfer Visual Language Models to Open-World Tracking Efficiently and Effectively](https://mlsanity.com/paper/2504.05141/) (2025)
- [evMLP: An Efficient Event-Driven MLP Architecture for Vision](https://mlsanity.com/paper/2507.01927/) (2025)
- [UBATrack: Spatio-Temporal State Space Model for General Multi-Modal Tracking](https://mlsanity.com/paper/2601.14799/) (2026)
- [Strip-Fusion: Spatiotemporal Fusion for Multispectral Pedestrian Detection](https://mlsanity.com/paper/2601.18008/) (2026)
- [Attention Is not Everything: Efficient Alternatives for Vision](https://mlsanity.com/paper/2604.17439/) (2026)

## Abstract (arXiv)

Token interaction operation is one of the core modules in MLP-based models to exchange and aggregate information between different spatial locations. However, the power of token interaction on the spatial dimension is highly dependent on the spatial resolution of the feature maps, which limits the model's expressive ability, especially in deep layers where the feature are down-sampled to a small spatial size. To address this issue, we present a novel method called Strip-MLP to enrich the token interaction power in three ways. Firstly, we introduce a new MLP paradigm called Strip MLP layer that allows the token to interact with other tokens in a cross-strip manner, enabling the tokens in a row (or column) to contribute to the information aggregations in adjacent but different strips of rows (or columns). Secondly, a Cascade Group Strip Mixing Module (CGSMM) is proposed to overcome the performance degradation caused by small spatial feature size. The module allows tokens to interact more effectively in the manners of within-patch and cross-patch, which is independent to the feature spatial size. Finally, based on the Strip MLP layer, we propose a novel Local Strip Mixing Module (LSMM) to boost the token interaction power in the local region. Extensive experiments demonstrate that Strip-MLP significantly improves the performance of MLP-based models on small datasets and obtains comparable or even better results on ImageNet. In particular, Strip-MLP models achieve higher average Top-1 accuracy than existing MLP-based models by +2.44% on Caltech-101 and +2.16% on CIFAR-100. The source codes will be available at~hrefhttps://github.com/Med-Process/StripMLPhttps://github.com/Med-Process/StripMLP.