mlsanityData to 14 Oct 2026

Rethinking the Function of Neurons in KANs

arXiv 2407.20667 ↗First posted 30 Jul 2024cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

The neurons of Kolmogorov-Arnold Networks (KANs) perform a simple summation motivated by the Kolmogorov-Arnold representation theorem, which asserts that sum is the only fundamental multivariate function. In this work, we investigate the potential for identifying an alternative multivariate function for KAN neurons that may offer increased practical utility. Our empirical research involves testing various multivariate functions in KAN neurons across a range of benchmark Machine Learning tasks. Our findings indicate that substituting the sum with the average function in KAN neurons results in significant performance enhancements compared to traditional KANs. Our study demonstrates that this minor modification contributes to the stability of training by confining the input to the spline within the effective range of the activation function. Our implementation and experiments are available at: urlhttps://github.com/Ghaith81/dropkan

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 28 of its 29 references that are in the map. First posted on arXiv 30 Jul 2024.

Builds on

28 of 29 references.

References · 28

  1. 2016Layer Normalizationcited by 5,669
  2. 2024KAN: Kolmogorov-Arnold Networkscited by 649
  3. 2024Kolmogorov-Arnold Networks are Radial Basis Function Networkscited by 128
  4. 2024Wav-KAN: Wavelet Kolmogorov-Arnold Networkscited by 122
  5. 2024Kolmogorov-Arnold Networks (KANs) for Time Series Analysiscited by 98
  6. 2024DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problemscited by 84
  7. 2024Convolutional Kolmogorov-Arnold Networkscited by 77
  8. 2024KAN or MLP: A Fairer Comparisoncited by 73
  9. 2024fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functionscited by 64
  10. 2024GKAN: Graph Kolmogorov-Arnold Networkscited by 53
  11. 2024A Temporal Kolmogorov-Arnold Transformer for Time Series Forecastingcited by 52
  12. 2024KAGNNs: Kolmogorov-Arnold Networks meet Graph Learningcited by 49
16 more
  1. 2024BSRBF-KAN: A combination of B-splines and Radial Basis Functions in Kolmogorov-Arnold Networkscited by 46
  2. 2024Enhancing Graph Collaborative Filtering with FourierKAN Feature Transformationcited by 42
  3. 2024Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Taskscited by 40
  4. 2024KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Searchcited by 37
  5. 2024rKAN: Rational Kolmogorov-Arnold Networkscited by 35
  6. 2024Unveiling the Power of Wavelets: A Wavelet-based Kolmogorov-Arnold Network for Hyperspectral Image Classificationcited by 34
  7. 2024Suitability of KANs for Computer Vision: A preliminary investigationcited by 34
  8. 2024Kolmogorov-Arnold Graph Neural Networkscited by 32
  9. 2024A Benchmarking Study of Kolmogorov-Arnold Networks on Tabular Datacited by 25
  10. 2024Exploring the Limitations of Kolmogorov-Arnold Networks in Classification: Insights to Software Training and Hardware Implementationcited by 22
  11. 2024Reduced Effectiveness of Kolmogorov-Arnold Networks on Functions with Noisecited by 17
  12. 2024iKAN: Global Incremental Learning with KAN for Human Activity Recognition Across Heterogeneous Datasetscited by 16
  13. 2024SineKAN: Kolmogorov-Arnold Networks Using Sinusoidal Activation Functionscited by 15
  14. 2024DropKAN: Regularizing KANs by masking post-activationscited by 14
  15. 2024Initial Investigation of Kolmogorov-Arnold Networks (KANs) as Feature Extractors for IMU Based Human Activity Recognitioncited by 13
  16. 2021Surrogate-Assisted Genetic Algorithm for Wrapper Feature Selectioncited by 3

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024FC-KAN: Function Combinations in Kolmogorov-Arnold Networkscited by 19
  2. 2024KAN we improve on HEP classification tasks? Kolmogorov-Arnold Networks applied to an LHC physics examplecited by 4
  3. 2025Physics-informed KAN PointNet: Deep learning for simultaneous solutions to inverse problems in incompressible flow on numerous irregular geometriescited by 3
  4. 2025KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networkscited by 1
  5. 2025Combinations of Fast Activation and Trigonometric Functions in Kolmogorov-Arnold Networkscited by 0