# RepMLPNet: Hierarchical Vision MLP with Re-parameterized Locality

arXiv: https://arxiv.org/abs/2112.11081 · HTML: https://mlsanity.com/paper/2112.11081/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 47 references that are in the map.
- First posted on arXiv 21 Dec 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Rethinking the Value of Network Pruning](https://mlsanity.com/paper/1810.05270/) (2018)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [Pay Attention to MLPs](https://mlsanity.com/paper/2105.08050/) (2021)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond](https://mlsanity.com/paper/1904.11492/) (2019)
- [ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks](https://mlsanity.com/paper/1908.03930/) (2019)
- [CycleMLP: A MLP-like Architecture for Dense Prediction](https://mlsanity.com/paper/2107.10224/) (2021)
- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [Fast Training of Convolutional Networks through FFTs](https://mlsanity.com/paper/1312.5851/) (2013)

## Built on by (most cited first, 30 of 32)

- [Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs](https://mlsanity.com/paper/2203.06717/) (2022)
- [More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity](https://mlsanity.com/paper/2207.03620/) (2022)
- [Towards Efficient Visual Adaption via Structural Re-parameterization](https://mlsanity.com/paper/2302.08106/) (2023)
- [UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition](https://mlsanity.com/paper/2311.15599/) (2023)
- [MogaNet: Multi-order Gated Aggregation Network](https://mlsanity.com/paper/2211.03295/) (2022)
- [Can CNNs Be More Robust Than Transformers?](https://mlsanity.com/paper/2206.03452/) (2022)
- [Re-parameterizing Your Optimizers rather than Architectures](https://mlsanity.com/paper/2205.15242/) (2022)
- [Sequencer: Deep LSTM for Image Classification](https://mlsanity.com/paper/2205.01972/) (2022)
- [Zero-shot Generalizable Incremental Learning for Vision-Language Object Detection](https://mlsanity.com/paper/2403.01680/) (2024)
- [Unlocking Deterministic Robustness Certification on ImageNet](https://mlsanity.com/paper/2301.12549/) (2023)
- [RIFormer: Keep Your Vision Backbone Effective While Removing Token Mixer](https://mlsanity.com/paper/2304.05659/) (2023)
- [Toward a Holistic Evaluation of Robustness in CLIP Models](https://mlsanity.com/paper/2410.01534/) (2024)
- [RefConv: Re-parameterized Refocusing Convolution for Powerful ConvNets](https://mlsanity.com/paper/2310.10563/) (2023)
- [SinGRAV: Learning a Generative Radiance Volume from a Single Natural Scene](https://mlsanity.com/paper/2210.01202/) (2022)
- [Are Large Kernels Better Teachers than Transformers for ConvNets?](https://mlsanity.com/paper/2305.19412/) (2023)
- [Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representations](https://mlsanity.com/paper/2410.08049/) (2024)
- [Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities](https://mlsanity.com/paper/2401.14405/) (2024)
- [The Unreasonable Effectiveness of Fully-Connected Layers for Low-Data Regimes](https://mlsanity.com/paper/2210.05657/) (2022)
- [FalconNet: Factorization for the Light-weight ConvNets](https://mlsanity.com/paper/2306.06365/) (2023)
- [Post-Training Quantization for Re-parameterization via Coarse & Fine Weight Splitting](https://mlsanity.com/paper/2312.10588/) (2023)
- [Aligning in a Compact Space: Contrastive Knowledge Distillation between Heterogeneous Architectures](https://mlsanity.com/paper/2405.18524/) (2024)
- [DSNet: A Novel Way to Use Atrous Convolutions in Semantic Segmentation](https://mlsanity.com/paper/2406.03702/) (2024)
- [CM-MLP: Cascade Multi-scale MLP with Axial Context Relation Encoder for Edge Segmentation of Medical Image](https://mlsanity.com/paper/2208.10701/) (2022)
- [Make Explicit Calibration Implicit: Calibrate Denoiser Instead of the Noise Model](https://mlsanity.com/paper/2308.03448/) (2023)
- [Conv-INR: Convolutional Implicit Neural Representation for Multimodal Visual Signals](https://mlsanity.com/paper/2406.04249/) (2024)
- [Reparameterized Multi-Resolution Convolutions for Long Sequence Modelling](https://mlsanity.com/paper/2408.09453/) (2024)
- [TriMLP: Revenge of a MLP-like Architecture in Sequential Recommendation](https://mlsanity.com/paper/2305.14675/) (2023)
- [Unifying Visual and Semantic Feature Spaces with Diffusion Models for Enhanced Cross-Modal Alignment](https://mlsanity.com/paper/2407.18854/) (2024)
- [Semantic-Space-Intervened Diffusive Alignment for Visual Classification](https://mlsanity.com/paper/2505.05721/) (2025)
- [A Class-wise Non-salient Region Generalized Framework for Video Semantic Segmentation](https://mlsanity.com/paper/2212.14154/) (2022)

## Abstract (arXiv)

Compared to convolutional layers, fully-connected (FC) layers are better at modeling the long-range dependencies but worse at capturing the local patterns, hence usually less favored for image recognition. In this paper, we propose a methodology, Locality Injection, to incorporate local priors into an FC layer via merging the trained parameters of a parallel conv kernel into the FC kernel. Locality Injection can be viewed as a novel Structural Re-parameterization method since it equivalently converts the structures via transforming the parameters. Based on that, we propose a multi-layer-perceptron (MLP) block named RepMLP Block, which uses three FC layers to extract features, and a novel architecture named RepMLPNet. The hierarchical design distinguishes RepMLPNet from the other concurrently proposed vision MLPs. As it produces feature maps of different levels, it qualifies as a backbone model for downstream tasks like semantic segmentation. Our results reveal that 1) Locality Injection is a general methodology for MLP models; 2) RepMLPNet has favorable accuracy-efficiency trade-off compared to the other MLPs; 3) RepMLPNet is the first MLP that seamlessly transfer to Cityscapes semantic segmentation. The code and models are available at https://github.com/DingXiaoH/RepMLP.