# Initialization Schemes for Kolmogorov-Arnold Networks: An Empirical Study

arXiv: https://arxiv.org/abs/2509.03417 · HTML: https://mlsanity.com/paper/2509.03417/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 3 Sep 2025.

## Builds on (most cited first, 20 of 20)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [All you need is a good init](https://mlsanity.com/paper/1511.06422/) (2015)
- [A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks](https://mlsanity.com/paper/2406.02917/) (2024)
- [DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problems](https://mlsanity.com/paper/2405.19143/) (2024)
- [Residual-based attention in physics-informed neural networks](https://mlsanity.com/paper/2307.00379/) (2023)
- [PirateNets: Physics-informed Deep Learning with Residual Adaptive Networks](https://mlsanity.com/paper/2402.00326/) (2024)
- [Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks](https://mlsanity.com/paper/2406.11045/) (2024)
- [Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2407.17611/) (2024)
- [KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search](https://mlsanity.com/paper/2406.17630/) (2024)
- [On the expressiveness and spectral bias of KANs](https://mlsanity.com/paper/2410.01803/) (2024)
- [A Benchmarking Study of Kolmogorov-Arnold Networks on Tabular Data](https://mlsanity.com/paper/2406.14529/) (2024)
- [Kolmogorov-Arnold PointNet: Deep learning for prediction of fluid fields on irregular geometries](https://mlsanity.com/paper/2408.02950/) (2024)
- [Deep Learning Alternatives of the Kolmogorov Superposition Theorem](https://mlsanity.com/paper/2410.01990/) (2024)
- [On the Robustness of Kolmogorov-Arnold Networks: An Adversarial Perspective](https://mlsanity.com/paper/2408.13809/) (2024)
- [Generalization Bounds and Model Complexity for Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.08026/) (2024)

## Built on by (most cited first, 11 of 11)

- [KANO: Kolmogorov-Arnold Neural Operator](https://mlsanity.com/paper/2509.16825/) (2025)
- [SINDy-KANs: Sparse identification of non-linear dynamics through Kolmogorov-Arnold networks](https://mlsanity.com/paper/2603.18548/) (2026)
- [Making Gaussian Kolmogorov-Arnold Networks Reliable and Accurate](https://mlsanity.com/paper/2604.21174/) (2026)
- [Optimized Architectures for Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2512.12448/) (2025)
- [Learning Nonlinear Heterogeneity in Physical Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2601.15340/) (2026)
- [A Dynamic Framework for Grid Adaptation in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2601.18672/) (2026)
- [Partition-of-Unity Gaussian Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2604.23599/) (2026)
- [KANs need curvature: penalties for compositional smoothness](https://mlsanity.com/paper/2605.02190/) (2026)
- [An adaptive framework for the axisymmetric pulsar magnetosphere using physics-informed Kolmogorov-Arnold networks](https://mlsanity.com/paper/2606.10686/) (2026)
- [LLN: Learnable Lens Networks for Parameter-Efficient Long-Horizon Dynamical Prediction](https://mlsanity.com/paper/2609.34493/) (2026)
- [KANLab: TensorFlow Playground for Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2610.20117/) (2026)

## Abstract (arXiv)

Kolmogorov-Arnold Networks (KANs) are a recently introduced neural architecture that replace fixed nonlinearities with trainable activation functions, offering enhanced flexibility and interpretability. While KANs have been applied successfully across scientific and machine learning tasks, their initialization strategies remain largely unexplored. In this work, we study initialization schemes for spline-based KANs, proposing two theory-driven approaches inspired by LeCun and Glorot, as well as an empirical power-law family with tunable exponents. Our evaluation combines large-scale grid searches on function fitting and forward PDE benchmarks, an analysis of training dynamics through the lens of the Neural Tangent Kernel, and evaluations on a subset of the Feynman dataset. Our findings indicate that the Glorot-inspired initialization significantly outperforms the baseline in parameter-rich models, while power-law initialization achieves the strongest performance overall, both across tasks and for architectures of varying size. All code and data accompanying this manuscript are publicly available at https://github.com/srigas/KAN_Initialization_Schemes.