mlsanityData to 7 Oct 2026

UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio, Video, Point Cloud, Time-Series and Image Recognition

arXiv 2311.15599 ↗First posted 27 Nov 2023cs.CV cs.AI cs.LGcited by 64

Not on the map. Why?

Abstract (arXiv)

Large-kernel convolutional neural networks (ConvNets) have recently received extensive research attention, but two unresolved and critical issues demand further investigation. 1) The architectures of existing large-kernel ConvNets largely follow the design principles of conventional ConvNets or transformers, while the architectural design for large-kernel ConvNets remains under-addressed. 2) As transformers have dominated multiple modalities, it remains to be investigated whether ConvNets also have a strong universal perception ability in domains beyond vision. In this paper, we contribute from two aspects. 1) We propose four architectural guidelines for designing large-kernel ConvNets, the core of which is to exploit the essential characteristics of large kernels that distinguish them from small kernels - they can see wide without going deep. Following such guidelines, our proposed large-kernel ConvNet shows leading performance in image recognition (ImageNet accuracy of 88.0%, ADE20K mIoU of 55.6%, and COCO box AP of 56.4%), demonstrating better performance and higher speed than the recent powerful competitors. 2) We discover large kernels are the key to unlocking the exceptional performance of ConvNets in domains where they were originally not proficient. With certain modality-related preprocessing approaches, the proposed model achieves state-of-the-art performance on time-series forecasting and audio recognition tasks even without modality-specific customization to the architecture. All the code and models are publicly available on GitHub and Huggingface.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 78 of its 101 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

78 of 101 references.

References · 78

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2017Mask R-CNNcited by 7,259
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2016Densely Connected Convolutional Networkscited by 6,691
  10. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  11. 2016Layer Normalizationcited by 5,579
  12. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
38 more
  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2017Squeeze-and-Excitation Networkscited by 4,763
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  6. 2014Going Deeper with Convolutionscited by 4,231
  7. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  11. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  12. 2017The Kinetics Human Action Video Datasetcited by 1,894
  13. 2018SlowFast Networks for Video Recognitioncited by 1,862
  14. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  15. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  16. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  17. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  18. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  19. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  20. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  21. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  22. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  23. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  24. 2020Designing Network Design Spacescited by 954
  25. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  26. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  27. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  28. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  29. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  30. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  31. 2021AST: Audio Spectrogram Transformercited by 634
  32. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  33. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
  34. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 549
  35. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  36. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  37. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  38. 2021MViTv2: Improved Multiscale Vision Transformers for Classification and Detectioncited by 516

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 28; 2025: 21; 2026: 14.

64 papers.

Sort:

By area:

  1. 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 83
  2. 2025OverLoCK: An Overview-first-Look-Closely-next ConvNet with Context-Mixing Dynamic Kernelscited by 24
  3. 2025Frequency Dynamic Convolution for Dense Image Predictioncited by 19
  4. 2025LSNet: See Large, Focus Smallcited by 15
  5. 2024Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Enginescited by 14
  6. 2025Strip R-CNN: Large Strip Convolution for Remote Sensing Object Detectioncited by 14
  7. 2024GrootVL: Tree Topology is All You Need in State Space Modelcited by 13
  8. 2024ScaleKD: Strong Vision Transformers Could Be Excellent Teacherscited by 11
  9. 2024RoadFormer+: Delivering RGB-X Scene Parsing through Scale-Aware Information Decoupling and Advanced Heterogeneous Feature Fusioncited by 9
  10. 2025DAMamba: Vision State Space Model with Dynamic Adaptive Scancited by 9
  11. 2024Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustnesscited by 8
  12. 2024Scaling Up Your Kernels: Large Kernel Design in ConvNets towards Universal Representationscited by 8
38 more
  1. 2024Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalitiescited by 7
  2. 2024Partial Large Kernel CNNs for Efficient Super-Resolutioncited by 7
  3. 2025Pre-Trained LLM is a Semantic-Aware and Generalizable Segmentation Boostercited by 7
  4. 2025DiCo: Revitalizing ConvNets for Scalable and Efficient Diffusion Modelingcited by 6
  5. 2024StarLKNet: Star Mixup with Large Kernel Networks for Palm Vein Identificationcited by 5
  6. 2024ShiftwiseConv: Small Convolutional Kernel with Large Kernel Effectcited by 4
  7. 2024Explore the Limits of Omni-modal Pretraining at Scalecited by 4
  8. 2024RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterizationcited by 4
  9. 2024DeBiFormer: Vision Transformer with Deformable Agent Bi-level Routing Attentioncited by 4
  10. 2024InteractiveVideo: User-Centric Controllable Video Generation with Synergistic Multimodal Instructionscited by 3
  11. 2024LKCell: Efficient Cell Nuclei Instance Segmentation with Large Convolution Kernelscited by 3
  12. 2025UniConvNet: Expanding Effective Receptive Field while Maintaining Asymptotically Gaussian Distribution for ConvNets of Any Scalecited by 3
  13. 2025AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perceptioncited by 3
  14. 2025RCTDistill: Cross-Modal Knowledge Distillation Framework for Radar-Camera 3D Object Detection with Temporal Fusioncited by 3
  15. 2025MMDrive: Interactive Scene Understanding Beyond Vision with Multi-representational Fusioncited by 3
  16. 2023DECO: Unleashing the Potential of ConvNets for Query-based Detection and Segmentationcited by 2
  17. 2024MiniMaxAD: A Lightweight Autoencoder for Feature-Rich Anomaly Detectioncited by 2
  18. 2024Reparameterized Multi-Resolution Convolutions for Long Sequence Modellingcited by 2
  19. 2024Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learningcited by 2
  20. 2024ContextDet: Temporal Action Detection with Adaptive Context Aggregationcited by 2
  21. 2025A Comprehensive Survey on Video Scene Parsing:Advances, Challenges, and Prospectscited by 2
  22. 2026SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMscited by 2
  23. 2024FFNet: MetaMixer-based Efficient Convolutional Mixer Designcited by 1
  24. 2024Real-Time 3D Occupancy Prediction via Geometric-Semantic Disentanglementcited by 1
  25. 2024GatedUniPose: A Novel Approach for Pose Estimation Combining UniRepLKNet and Gated Convolutioncited by 1
  26. 2025TinyDrive: Multiscale Visual Question Answering with Selective Token Routing for Autonomous Drivingcited by 1
  27. 2025A2Mamba: Attention-augmented State Space Models for Visual Recognitioncited by 1
  28. 2026Uncertainty-aware Spatial-Frequency Registration and Fusion for Infrared and Visible Imagescited by 1
  29. 2026EchoSR: Efficient Context Harnessing for Lightweight Image Super-Resolutioncited by 1
  30. 2024Siamese Multiple Attention Temporal Convolution Networks for Human Mobility Signature Identificationcited by 0
  31. 2024CoMiX: Cross-Modal Fusion with Deformable Convolutions for HSI-X Semantic Segmentationcited by 0
  32. 2024DREAM: Domain-agnostic Reverse Engineering Attributes of Black-box Modelcited by 0
  33. 2024Learning Dynamic Local Context Representations for Infrared Small Target Detectioncited by 0
  34. 2024RecConv: Efficient Recursive Convolutions for Multi-Frequency Representationscited by 0
  35. 2025SFMNet: Sparse Focal Modulation for 3D Object Detectioncited by 0
  36. 2025GmNet: Revisiting Gating Mechanisms From A Frequency Viewcited by 0
  37. 2025LSP-ST: Ladder Shape-Biased Side-Tuning for Robust Infrared Small Target Detectioncited by 0
  38. 2025Progressive Multimodal Interaction Network for Reliable Quantification of Fish Feeding Intensity in Aquaculturecited by 0

50 most cited of 64 All 64 (JSON)