mlsanityData to 7 Oct 2026

DropKAN: Regularizing KANs by masking post-activations

arXiv 2407.13044 ↗First posted 17 Jul 2024cs.LG cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

We propose DropKAN (Dropout Kolmogorov-Arnold Networks) a regularization method that prevents co-adaptation of activation function weights in Kolmogorov-Arnold Networks (KANs). DropKAN functions by embedding the drop mask directly within the KAN layer, randomly masking the outputs of some activations within the KANs' computation graph. We show that this simple procedure that require minimal coding effort has a regularizing effect and consistently lead to better generalization of KANs. We analyze the adaptation of the standard Dropout with KANs and demonstrate that Dropout applied to KANs' neurons can lead to unpredictable behavior in the feedforward pass. We carry an empirical study with real world Machine Learning datasets to validate our findings. Our results suggest that DropKAN is consistently a better alternative to using standard Dropout with KANs, and improves the generalization performance of KANs. Our implementation of DropKAN is available at: urlhttps://github.com/Ghaith81/dropkan.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

28 of 34 references.

References · 28

  1. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  2. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  3. 2018DropBlock: A regularization method for convolutional networkscited by 261
  4. 2014Efficient Object Localization Using Convolutional Networkscited by 226
  5. 2013Maxout Networkscited by 186
  6. 2024Kolmogorov-Arnold Networks are Radial Basis Function Networkscited by 125
  7. 2024Wav-KAN: Wavelet Kolmogorov-Arnold Networkscited by 118
  8. 2024Kolmogorov-Arnold Networks (KANs) for Time Series Analysiscited by 97
  9. 2024DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problemscited by 82
  10. 2024Convolutional Kolmogorov-Arnold Networkscited by 76
  11. 2024fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functionscited by 60
  12. 2024GKAN: Graph Kolmogorov-Arnold Networkscited by 52
16 more
  1. 2024A Temporal Kolmogorov-Arnold Transformer for Time Series Forecastingcited by 50
  2. 2024KAGNNs: Kolmogorov-Arnold Networks meet Graph Learningcited by 48
  3. 2024BSRBF-KAN: A combination of B-splines and Radial Basis Functions in Kolmogorov-Arnold Networkscited by 43
  4. 2024Enhancing Graph Collaborative Filtering with FourierKAN Feature Transformationcited by 41
  5. 2024Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Taskscited by 39
  6. 2024KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Searchcited by 36
  7. 2024Unveiling the Power of Wavelets: A Wavelet-based Kolmogorov-Arnold Network for Hyperspectral Image Classificationcited by 33
  8. 2024Suitability of KANs for Computer Vision: A preliminary investigationcited by 33
  9. 2024rKAN: Rational Kolmogorov-Arnold Networkscited by 31
  10. 2024Kolmogorov-Arnold Graph Neural Networkscited by 31
  11. 2020Regularizing Meta-Learning via Gradient Dropoutcited by 15
  12. 2024SineKAN: Kolmogorov-Arnold Networks Using Sinusoidal Activation Functionscited by 15
  13. 2024iKAN: Global Incremental Learning with KAN for Human Activity Recognition Across Heterogeneous Datasetscited by 14
  14. 2024Initial Investigation of Kolmogorov-Arnold Networks (KANs) as Feature Extractors for IMU Based Human Activity Recognitioncited by 12
  15. 2021Surrogate-Assisted Genetic Algorithm for Wrapper Feature Selectioncited by 3
  16. 2024Rolling the dice for better deep learning performance: A study of randomness techniques in deep neural networkscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 5; 2026: 2.

14 papers.

Sort:

  1. 2024KAN 2.0: Kolmogorov-Arnold Networks Meet Sciencecited by 116
  2. 2024A Survey on Kolmogorov-Arnold Networkcited by 67
  3. 2024FC-KAN: Function Combinations in Kolmogorov-Arnold Networkscited by 18
  4. 2024Generalization Bounds and Model Complexity for Kolmogorov-Arnold Networkscited by 11
  5. 2024KAN versus MLP on Irregular or Noisy Functionscited by 7
  6. 2024Rethinking the Function of Neurons in KANscited by 5
  7. 2024KAN we improve on HEP classification tasks? Kolmogorov-Arnold Networks applied to an LHC physics examplecited by 4
  8. 2025Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learningcited by 3
  9. 2025Interpretable Clinical Classification with Kolmogorov-Arnold Networkscited by 3
  10. 2025KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networkscited by 1
  11. 2026TruKAN: Towards More Efficient Kolmogorov-Arnold Networks Using Truncated Power Functionscited by 1
  12. 2025HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagationcited by 0
2 more
  1. 2025Combinations of Fast Activation and Trigonometric Functions in Kolmogorov-Arnold Networkscited by 0
  2. 2026KAN-MLP-Mixer: A comprehensive investigation of the usage of Kolmogorov-Arnold Networks (KANs) for improving IMU-based Human Activity Recognitioncited by 0