mlsanityData to 7 Oct 2026

On the expressiveness and spectral bias of KANs

arXiv 2410.01803 ↗First posted 2 Oct 2024cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

Kolmogorov-Arnold Networks (KAN) citeliu2024kan were very recently proposed as a potential alternative to the prevalent architectural backbone of many deep learning models, the multi-layer perceptron (MLP). KANs have seen success in various tasks of AI for science, with their empirical efficiency and accuracy demostrated in function regression, PDE solving, and many more scientific problems. In this article, we revisit the comparison of KANs and MLPs, with emphasis on a theoretical perspective. On the one hand, we compare the representation and approximation capabilities of KANs and MLPs. We establish that MLPs can be represented using KANs of a comparable size. This shows that the approximation and representation capabilities of KANs are at least as good as MLPs. Conversely, we show that KANs can be represented using MLPs, but that in this representation the number of parameters increases by a factor of the KAN grid size. This suggests that KANs with a large grid size may be more efficient than MLPs at approximating certain functions. On the other hand, from the perspective of learning and optimization, we study the spectral bias of KANs compared with MLPs. We demonstrate that KANs are less biased toward low frequencies than MLPs. We highlight that the multi-level learning feature specific to KANs, i.e. grid extension of splines, improves the learning process for high-frequency components. Detailed comparisons with different choices of depth, width, and grid sizes of KANs are made, shedding some light on how to choose the hyperparameters in practice.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 62 of its 86 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

62 of 86 references.

References · 62

  1. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  2. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  3. 2018On the Spectral Bias of Neural Networkscited by 1,321
  4. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  5. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  6. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  7. 2016Error bounds for approximations with deep ReLU networkscited by 567
  8. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  9. 2019Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networkscited by 360
  10. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
  11. 2024U-KAN Makes Strong Backbone for Medical Image Segmentation and Generationcited by 176
  12. 2018Training behavior of deep neural network in frequency domaincited by 174
38 more
  1. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 165
  2. 2020Deep Network Approximation for Smooth Functionscited by 163
  3. 2020Frequency Bias in Neural Networks for Input of Non-Uniform Densitycited by 140
  4. 2019The Convergence Rate of Neural Networks for Learned Functions of Different Frequenciescited by 134
  5. 2024Kolmogorov-Arnold Networks are Radial Basis Function Networkscited by 125
  6. 2024Wav-KAN: Wavelet Kolmogorov-Arnold Networkscited by 118
  7. 2024KAN 2.0: Kolmogorov-Arnold Networks Meet Sciencecited by 116
  8. 2024Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximationcited by 115
  9. 2024Challenges in Training PINNs: A Loss Landscape Perspectivecited by 109
  10. 2024A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networkscited by 103
  11. 2024Kolmogorov-Arnold Networks (KANs) for Time Series Analysiscited by 97
  12. 2021Optimal Approximation Rate of ReLU Networks in terms of Width and Depthcited by 95
  13. 2019The phase diagram of approximation rates for deep neural networkscited by 86
  14. 2021Sharp Bounds on the Approximation Rates, Metric Entropy, and n-widths of Shallow Neural Networkscited by 86
  15. 2016Approximation by Combinations of ReLU and Squared ReLU Ridge Functions with ℓ1 and ℓ0 Controlscited by 84
  16. 2024DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problemscited by 82
  17. 2024Convolutional Kolmogorov-Arnold Networkscited by 76
  18. 2024TKAN: Temporal Kolmogorov-Arnold Networkscited by 70
  19. 2024Kolmogorov-Arnold Networks for Time Series: Bridging Predictive Power and Interpretabilitycited by 69
  20. 2024fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functionscited by 60
  21. 2024GKAN: Graph Kolmogorov-Arnold Networkscited by 52
  22. 2021Seeing Implicit Neural Representations as Fourier Seriescited by 50
  23. 2022Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spacescited by 50
  24. 2024A Temporal Kolmogorov-Arnold Transformer for Time Series Forecastingcited by 50
  25. 2024KAGNNs: Kolmogorov-Arnold Networks meet Graph Learningcited by 48
  26. 2024Kolmogorov-Arnold Network for Satellite Image Classification in Remote Sensingcited by 46
  27. 2024BSRBF-KAN: A combination of B-splines and Radial Basis Functions in Kolmogorov-Arnold Networkscited by 43
  28. 2024Enhancing Graph Collaborative Filtering with FourierKAN Feature Transformationcited by 41
  29. 2024Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Taskscited by 39
  30. 2024Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networkscited by 37
  31. 2021Spectral Bias in Practice: The Role of Function Frequency in Generalizationcited by 36
  32. 2024KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Searchcited by 36
  33. 2024Exploring the Potential of Polynomial Basis Functions in Kolmogorov-Arnold Networks: A Comparative Study of Different Groups of Polynomialscited by 33
  34. 2024Unveiling the Power of Wavelets: A Wavelet-based Kolmogorov-Arnold Network for Hyperspectral Image Classificationcited by 33
  35. 2024Suitability of KANs for Computer Vision: A preliminary investigationcited by 33
  36. 2024GraphKAN: Enhancing Feature Extraction with Graph Kolmogorov Arnold Networkscited by 31
  37. 2024rKAN: Rational Kolmogorov-Arnold Networkscited by 31
  38. 2024Kolmogorov-Arnold Graph Neural Networkscited by 31

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 18; 2026: 14.

36 papers.

Sort:

  1. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  2. 2025KANEL'E: Kolmogorov-Arnold Networks for Efficient LUT-based Evaluationcited by 13
  3. 2024Predicting Crack Nucleation and Propagation in Brittle Materials Using Deep Operator Networks with Diverse Trunk Architecturescited by 12
  4. 2025Binned Spectral Power Loss for Improved Prediction of Chaotic Systemscited by 10
  5. 2025Initialization Schemes for Kolmogorov-Arnold Networks: An Empirical Studycited by 10
  6. 2024Learnable Activation Functions in Physics-Informed Neural Networks for Solving Partial Differential Equationscited by 7
  7. 2025High precision PINNs in unbounded domains: application to singularity formulation in PDEscited by 6
  8. 2025A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learningcited by 6
  9. 2025KANO: Kolmogorov-Arnold Neural Operatorcited by 6
  10. 2024MvKeTR: Chest CT Report Generation with Multi-View Perception and Knowledge Enhancementcited by 4
  11. 2025Improving Memory Efficiency for Training KANs via Meta Learningcited by 3
  12. 2025One model to solve them all: 2BSDE families via neural operatorscited by 3
24 more
  1. 2025Approximation Rates in Besov Norms and Sample-Complexity of Kolmogorov-Arnold Networks with Residual Connectionscited by 2
  2. 2025Training Deep Physics-Informed Kolmogorov-Arnold Networkscited by 2
  3. 2026Free-RBF-KAN: Kolmogorov-Arnold Networks with Adaptive Radial Basis Functions for Efficient Function Learningcited by 2
  4. 2025Low Tensor-Rank Adaptation of Kolmogorov--Arnold Networkscited by 1
  5. 2025INR-Bench: A Unified Benchmark for Implicit Neural Representations in Multi-Domain Regression and Reconstructioncited by 1
  6. 2026A Dynamic Framework for Grid Adaptation in Kolmogorov-Arnold Networkscited by 1
  7. 2026Time series forecasting with Hahn Kolmogorov-Arnold networkscited by 1
  8. 2026Optimization, Generalization and Differential Privacy Bounds for Gradient Descent on Kolmogorov-Arnold Networkscited by 1
  9. 2026Fourier Feature Pyramids for Physics-Informed Neural Networkscited by 1
  10. 2026EML Trees Are Universal Approximatorscited by 1
  11. 2026RecKAN: Kolmogorov-Arnold Networks with a Learnable Recursive Polynomial Basiscited by 1
  12. 2025HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagationcited by 0
  13. 2025Enhancing Physics-Informed Neural Networks with a Hybrid Parallel Kolmogorov-Arnold and MLP Architecturecited by 0
  14. 2025LuKAN: A Kolmogorov-Arnold Network Framework for 3D Human Motion Predictioncited by 0
  15. 2025AR-KAN: Autoregressive-Weight-Enhanced Kolmogorov-Arnold Network for Time Series Forecastingcited by 0
  16. 2025FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharingcited by 0
  17. 2025Adaptive graph Kolmogorov-Arnold network for 3D human pose estimationcited by 0
  18. 2026ABFR-KAN: Kolmogorov-Arnold Networks for Functional Brain Analysiscited by 0
  19. 2026Autocorrelation Reintroduces Spectral Bias in KANs for Time Series Forecastingcited by 0
  20. 2026KAPLAN: Kolmogorov-Arnold Prognostic Learnable Activation Networks for Survival Analysiscited by 0
  21. 2026An adaptive framework for the axisymmetric pulsar magnetosphere using physics-informed Kolmogorov-Arnold networkscited by 0
  22. 2026PG-KINN: A Physics-Informed Petrov-Galerkin Kolmogorov-Arnold Network for Solving Forward and Inverse PDEscited by 0
  23. 2026Fast, accurate, and differentiable: a neural-network surrogate for NRSur7dq4 precessing binary black hole waveformscited by 0
  24. 2026Fisher Simplicity in Kolmogorov-Arnold Networks and Multilayer Perceptronscited by 0