# PeLK: Parameter-efficient Large Kernel ConvNets with Peripheral Convolution

arXiv: https://arxiv.org/abs/2403.07589 · HTML: https://mlsanity.com/paper/2403.07589/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 59 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network](https://mlsanity.com/paper/1703.02719/) (2017)
- [Patches Are All You Need?](https://mlsanity.com/paper/2201.09792/) (2022)
- [More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity](https://mlsanity.com/paper/2207.03620/) (2022)
- [InceptionNeXt: When Inception Meets ConvNeXt](https://mlsanity.com/paper/2303.16900/) (2023)
- [LargeKernel3D: Scaling up Kernels in 3D Sparse CNNs](https://mlsanity.com/paper/2206.10555/) (2022)

## Built on by (most cited first, 28 of 28)

- [OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernels](https://mlsanity.com/paper/2502.20087/) (2025)
- [SparX: A Sparse Cross-Layer Connection Mechanism for Hierarchical Vision Mamba and Transformer Networks](https://mlsanity.com/paper/2409.09649/) (2024)
- [Strip R-CNN: Large Strip Convolution for Remote Sensing Object Detection](https://mlsanity.com/paper/2501.03775/) (2025)
- [Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers](https://mlsanity.com/paper/2507.15833/) (2025)
- [Frequency-Dynamic Attention Modulation for Dense Prediction](https://mlsanity.com/paper/2507.12006/) (2025)
- [Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustness](https://mlsanity.com/paper/2407.08972/) (2024)
- [Emulating Self-attention with Convolution for Efficient Image Super-Resolution](https://mlsanity.com/paper/2503.06671/) (2025)
- [Segment This Thing: Foveated Tokenization for Efficient Point-Prompted Segmentation](https://mlsanity.com/paper/2506.11131/) (2025)
- [RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterization](https://mlsanity.com/paper/2406.16004/) (2024)
- [Optimizing Local-Global Dependencies for Accurate 3D Human Pose Estimation](https://mlsanity.com/paper/2412.19676/) (2024)
- [S2AFormer: Strip Self-Attention for Efficient Vision Transformer](https://mlsanity.com/paper/2505.22195/) (2025)
- [Spatial Frequency Modulation for Semantic Segmentation](https://mlsanity.com/paper/2507.11893/) (2025)
- [UniConvNet: Expanding Effective Receptive Field while Maintaining Asymptotically Gaussian Distribution for ConvNets of Any Scale](https://mlsanity.com/paper/2508.09000/) (2025)
- [Large Kernel Modulation Network for Efficient Image Super-Resolution](https://mlsanity.com/paper/2508.11893/) (2025)
- [ContextDet: Temporal Action Detection with Adaptive Context Aggregation](https://mlsanity.com/paper/2410.15279/) (2024)
- [Efficient Visual Representation Learning with Heat Conduction Equation](https://mlsanity.com/paper/2408.05901/) (2024)
- [A Retrospective Systematic Study on Hierarchical Sparse Query Transformer-assisted Ultrasound Screening for Early Hepatocellular Carcinoma](https://mlsanity.com/paper/2502.03772/) (2025)
- [A2Mamba: Attention-augmented State Space Models for Visual Recognition](https://mlsanity.com/paper/2507.16624/) (2025)
- [Learning Dynamic Local Context Representations for Infrared Small Target Detection](https://mlsanity.com/paper/2412.17401/) (2024)
- [RecConv: Efficient Recursive Convolutions for Multi-Frequency Representations](https://mlsanity.com/paper/2412.19628/) (2024)
- [Spatially-Adaptive Gradient Re-parameterization for 3D Large Kernel Optimization](https://mlsanity.com/paper/2505.19603/) (2025)
- [InceptionMamba: An Efficient Hybrid Network with Large Band Convolution and Bottleneck Mamba](https://mlsanity.com/paper/2506.08735/) (2025)
- [LKFMixer: Exploring Large Kernel Feature For Efficient Image Super-Resolution](https://mlsanity.com/paper/2508.11391/) (2025)
- [MKSNet: Advanced Small Object Detection in Remote Sensing Imagery with Multi-Kernel and Dual Attention Mechanisms](https://mlsanity.com/paper/2512.03640/) (2025)
- [PFGNet: A Fully Convolutional Frequency-Guided Peripheral Gating Network for Efficient Spatiotemporal Predictive Learning](https://mlsanity.com/paper/2602.20537/) (2026)
- [SFFNet: Synergistic Feature Fusion Network With Dual-Domain Edge Enhancement for UAV Image Object Detection](https://mlsanity.com/paper/2604.03176/) (2026)
- [KAConvNet: Kolmogorov-Arnold Convolutional Networks for Vision Recognition](https://mlsanity.com/paper/2604.23320/) (2026)
- [Group-Shared Low-Rank Approximation for Mobile-Efficient Pointwise Convolutions in Large-Kernel CNNs](https://mlsanity.com/paper/2608.26069/) (2026)

## Abstract (arXiv)

Recently, some large kernel convnets strike back with appealing performance and efficiency. However, given the square complexity of convolution, scaling up kernels can bring about an enormous amount of parameters and the proliferated parameters can induce severe optimization problem. Due to these issues, current CNNs compromise to scale up to 51x51 in the form of stripe convolution (i.e., 51x5 + 5x51) and start to saturate as the kernel size continues growing. In this paper, we delve into addressing these vital issues and explore whether we can continue scaling up kernels for more performance gains. Inspired by human vision, we propose a human-like peripheral convolution that efficiently reduces over 90% parameter count of dense grid convolution through parameter sharing, and manage to scale up kernel size to extremely large. Our peripheral convolution behaves highly similar to human, reducing the complexity of convolution from O(K^2) to O(logK) without backfiring performance. Built on this, we propose Parameter-efficient Large Kernel Network (PeLK). Our PeLK outperforms modern vision Transformers and ConvNet architectures like Swin, ConvNeXt, RepLKNet and SLaK on various vision tasks including ImageNet classification, semantic segmentation on ADE20K and object detection on MS COCO. For the first time, we successfully scale up the kernel size of CNNs to an unprecedented 101x101 and demonstrate consistent improvements.