# DropKAN: Regularizing KANs by masking post-activations

arXiv: https://arxiv.org/abs/2407.13044 · HTML: https://mlsanity.com/paper/2407.13044/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 34 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [DropBlock: A regularization method for convolutional networks](https://mlsanity.com/paper/1810.12890/) (2018)
- [Efficient Object Localization Using Convolutional Networks](https://mlsanity.com/paper/1411.4280/) (2014)
- [Maxout Networks](https://mlsanity.com/paper/1302.4389/) (2013)
- [Kolmogorov-Arnold Networks are Radial Basis Function Networks](https://mlsanity.com/paper/2405.06721/) (2024)
- [Wav-KAN: Wavelet Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.12832/) (2024)
- [Kolmogorov-Arnold Networks (KANs) for Time Series Analysis](https://mlsanity.com/paper/2405.08790/) (2024)
- [DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problems](https://mlsanity.com/paper/2405.19143/) (2024)
- [Convolutional Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.13155/) (2024)
- [fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functions](https://mlsanity.com/paper/2406.07456/) (2024)
- [GKAN: Graph Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.06470/) (2024)
- [A Temporal Kolmogorov-Arnold Transformer for Time Series Forecasting](https://mlsanity.com/paper/2406.02486/) (2024)
- [KAGNNs: Kolmogorov-Arnold Networks meet Graph Learning](https://mlsanity.com/paper/2406.18380/) (2024)
- [BSRBF-KAN: A combination of B-splines and Radial Basis Functions in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.11173/) (2024)
- [Enhancing Graph Collaborative Filtering with FourierKAN Feature Transformation](https://mlsanity.com/paper/2406.01034/) (2024)
- [Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Tasks](https://mlsanity.com/paper/2406.14916/) (2024)
- [KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search](https://mlsanity.com/paper/2406.17630/) (2024)
- [rKAN: Rational Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.14495/) (2024)
- [Unveiling the Power of Wavelets: A Wavelet-based Kolmogorov-Arnold Network for Hyperspectral Image Classification](https://mlsanity.com/paper/2406.07869/) (2024)
- [Suitability of KANs for Computer Vision: A preliminary investigation](https://mlsanity.com/paper/2406.09087/) (2024)
- [Kolmogorov-Arnold Graph Neural Networks](https://mlsanity.com/paper/2406.18354/) (2024)
- [iKAN: Global Incremental Learning with KAN for Human Activity Recognition Across Heterogeneous Datasets](https://mlsanity.com/paper/2406.01646/) (2024)
- [Regularizing Meta-Learning via Gradient Dropout](https://mlsanity.com/paper/2004.05859/) (2020)
- [SineKAN: Kolmogorov-Arnold Networks Using Sinusoidal Activation Functions](https://mlsanity.com/paper/2407.04149/) (2024)
- [Initial Investigation of Kolmogorov-Arnold Networks (KANs) as Feature Extractors for IMU Based Human Activity Recognition](https://mlsanity.com/paper/2406.11914/) (2024)
- [Surrogate-Assisted Genetic Algorithm for Wrapper Feature Selection](https://mlsanity.com/paper/2111.09074/) (2021)
- [Rolling the dice for better deep learning performance: A study of randomness techniques in deep neural networks](https://mlsanity.com/paper/2404.03992/) (2024)

## Built on by (most cited first, 14 of 14)

- [KAN 2.0: Kolmogorov-Arnold Networks Meet Science](https://mlsanity.com/paper/2408.10205/) (2024)
- [A Survey on Kolmogorov-Arnold Network](https://mlsanity.com/paper/2411.06078/) (2024)
- [FC-KAN: Function Combinations in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2409.01763/) (2024)
- [Generalization Bounds and Model Complexity for Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.08026/) (2024)
- [KAN versus MLP on Irregular or Noisy Functions](https://mlsanity.com/paper/2408.07906/) (2024)
- [Rethinking the Function of Neurons in KANs](https://mlsanity.com/paper/2407.20667/) (2024)
- [KAN we improve on HEP classification tasks? Kolmogorov-Arnold Networks applied to an LHC physics example](https://mlsanity.com/paper/2408.02743/) (2024)
- [Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learning](https://mlsanity.com/paper/2503.21419/) (2025)
- [Interpretable Clinical Classification with Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2509.16750/) (2025)
- [KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networks](https://mlsanity.com/paper/2501.13456/) (2025)
- [TruKAN: Towards More Efficient Kolmogorov-Arnold Networks Using Truncated Power Functions](https://mlsanity.com/paper/2602.03879/) (2026)
- [HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagation](https://mlsanity.com/paper/2501.18199/) (2025)
- [Combinations of Fast Activation and Trigonometric Functions in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2508.11876/) (2025)
- [KAN-MLP-Mixer: A comprehensive investigation of the usage of Kolmogorov-Arnold Networks (KANs) for improving IMU-based Human Activity Recognition](https://mlsanity.com/paper/2605.19031/) (2026)

## Abstract (arXiv)

We propose DropKAN (Dropout Kolmogorov-Arnold Networks) a regularization method that prevents co-adaptation of activation function weights in Kolmogorov-Arnold Networks (KANs). DropKAN functions by embedding the drop mask directly within the KAN layer, randomly masking the outputs of some activations within the KANs' computation graph. We show that this simple procedure that require minimal coding effort has a regularizing effect and consistently lead to better generalization of KANs. We analyze the adaptation of the standard Dropout with KANs and demonstrate that Dropout applied to KANs' neurons can lead to unpredictable behavior in the feedforward pass. We carry an empirical study with real world Machine Learning datasets to validate our findings. Our results suggest that DropKAN is consistently a better alternative to using standard Dropout with KANs, and improves the generalization performance of KANs. Our implementation of DropKAN is available at: urlhttps://github.com/Ghaith81/dropkan.