mlsanityData to 7 Oct 2026

Residual Kolmogorov-Arnold Network for Enhanced Deep Learning

arXiv 2410.05500 ↗First posted 7 Oct 2024cs.CV cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Despite their immense success, deep convolutional neural networks (CNNs) can be difficult to optimize and costly to train due to hundreds of layers within the network depth. Conventional convolutional operations are fundamentally limited by their linear nature along with fixed activations, where many layers are needed to learn meaningful patterns in data. Because of the sheer size of these networks, this approach is simply computationally inefficient, and poses overfitting or gradient explosion risks, especially in small datasets. As a result, we introduce a "plug-in" module, called Residual Kolmogorov-Arnold Network (RKAN). Our module is highly compact, so it can be easily added into any stage (level) of traditional deep networks, where it learns to integrate supportive polynomial feature transformations to existing convolutional frameworks. RKAN offers consistent improvements over baseline models in different vision tasks and widely tested benchmarks, accomplishing cutting-edge performance on them.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 65 of its 91 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

65 of 91 references.

References · 65

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2017Mask R-CNNcited by 7,259
  8. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  9. 2016Densely Connected Convolutional Networkscited by 6,691
  10. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  11. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  12. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
38 more
  1. 2017Squeeze-and-Excitation Networkscited by 4,763
  2. 2022A ConvNet for the 2020scited by 3,545
  3. 2016Wide Residual Networkscited by 3,057
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  5. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  6. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  7. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  8. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  9. 2013Describing Textures in the Wildcited by 2,110
  10. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  11. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  12. 2014How transferable are features in deep neural networks?cited by 1,216
  13. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  14. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  15. 2020Designing Network Design Spacescited by 954
  16. 2017A Closer Look at Memorization in Deep Networkscited by 936
  17. 2018Path Aggregation Network for Instance Segmentationcited by 796
  18. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  19. 2015Cyclical Learning Rates for Training Neural Networkscited by 627
  20. 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 572
  21. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  22. 2018Do Better ImageNet Models Transfer Better?cited by 560
  23. 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 549
  24. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  25. 2019Res2Net: A New Multi-scale Backbone Architecturecited by 468
  26. 2020ResNeSt: Split-Attention Networkscited by 445
  27. 2017Deep Layer Aggregationcited by 436
  28. 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 430
  29. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 413
  30. 2019CSPNet: A New Backbone that can Enhance Learning Capability of CNNcited by 381
  31. 2018Rethinking ImageNet Pre-trainingcited by 376
  32. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  33. 2016On the Expressive Power of Deep Neural Networkscited by 302
  34. 2019A Survey of the Recent Architectures of Deep Convolutional Neural Networkscited by 211
  35. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  36. 2016Deep Pyramidal Residual Networkscited by 163
  37. 2023FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterizationcited by 161
  38. 2024Kolmogorov-Arnold Networks are Radial Basis Function Networkscited by 125

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 1.

5 papers.

  1. 2024A Survey on Kolmogorov-Arnold Networkcited by 67
  2. 2025Interpretable Clinical Classification with Kolmogorov-Arnold Networkscited by 3
  3. 2025CausalKANs: interpretable treatment effect estimation with Kolmogorov-Arnold networkscited by 2
  4. 2025HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagationcited by 0
  5. 2026Clifford Kolmogorov-Arnold Networkscited by 0