mlsanityData to 14 Oct 2026

Leveraging KANs for Expedient Training of Multichannel MLPs via Preconditioning and Geometric Refinement

arXiv 2505.18131 ↗First posted 23 May 2025cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Multilayer perceptrons (MLPs) are a workhorse machine learning architecture, used in a variety of modern deep learning frameworks. However, recently Kolmogorov-Arnold Networks (KANs) have become increasingly popular due to their success on a range of problems, particularly for scientific machine learning tasks. In this paper, we exploit the relationship between KANs and multichannel MLPs to gain structural insight into how to train MLPs faster. We demonstrate the KAN basis (1) provides geometric localized support, and (2) acts as a preconditioned descent in the ReLU basis, overall resulting in expedited training and improved accuracy. Our results show the equivalence between free-knot spline KAN architectures, and a class of MLPs that are refined geometrically along the channel dimension of each weight tensor. We exploit this structural equivalence to define a hierarchical refinement scheme that dramatically accelerates training of the multi-channel MLP architecture. We show further accuracy improvements can be had by allowing the 1D locations of the spline knots to be trained simultaneously with the weights. These advances are demonstrated on a range of benchmark examples for regression and scientific machine learning.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 36 of its 71 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

36 of 71 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  3. 2016Layer Normalizationcited by 5,669
  4. 2018On the Spectral Bias of Neural Networkscited by 1,341
  5. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  6. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  7. 2023An Expert's Guide to Training Physics-informed Neural Networkscited by 126
  8. 2024Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximationcited by 118
  9. 2024KAN 2.0: Kolmogorov-Arnold Networks Meet Sciencecited by 118
  10. 2024A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networkscited by 104
  11. 2024Kolmogorov-Arnold Networks (KANs) for Time Series Analysiscited by 98
  12. 2024DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problemscited by 84
24 more
  1. 2024KAN or MLP: A Fairer Comparisoncited by 73
  2. 2024From PINNs to PIKANs: Recent Advances in Physics-Informed Machine Learningcited by 73
  3. 2024PirateNets: Physics-informed Deep Learning with Residual Adaptive Networkscited by 72
  4. 2024A Survey on Kolmogorov-Arnold Networkcited by 68
  5. 2024KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physicscited by 59
  6. 2025Gradient Alignment in Physics-informed Neural Networks: A Second-Order Optimization Perspectivecited by 56
  7. 2024KAGNNs: Kolmogorov-Arnold Networks meet Graph Learningcited by 49
  8. 2024Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Taskscited by 40
  9. 2024Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networkscited by 38
  10. 2016Reliably Learning the ReLU in Polynomial Timecited by 36
  11. 2024Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problemscited by 35
  12. 2024ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLUcited by 33
  13. 2022On the Activation Function Dependence of the Spectral Bias of Neural Networkscited by 28
  14. 2024Local Feature Matching Using Deep Learning: A Surveycited by 26
  15. 2024Physics Informed Kolmogorov-Arnold Neural Networks for Dynamical Analysis via Efficent-KAN and WAV-KANcited by 25
  16. 2024Kolmogorov-Arnold Networks: A Critical Assessment of Claims, Performance, and Practical Viabilitycited by 23
  17. 2024Kolmogorov-Arnold Networks (KAN) for Time Series Classification and Robust Analysiscited by 16
  18. 2024PowerMLP: An Efficient Version of KANcited by 14
  19. 2024Higher-order-ReLU-KANs (HRKANs) for solving physics-informed neural networks (PINNs) more accurately, robustly and fastercited by 13
  20. 2024SPIKANs: Separable Physics-Informed Kolmogorov-Arnold Networkscited by 12
  21. 2024Multifidelity Kolmogorov-Arnold Networkscited by 9
  22. 2018Detecting Dead Weights and Units in Neural Networkscited by 7
  23. 2024A preliminary study on continual learning in computer vision using Kolmogorov-Arnold Networkscited by 7
  24. 2024MLP-KAN: Unifying Deep Representation and Function Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Free-RBF-KAN: Kolmogorov-Arnold Networks with Adaptive Radial Basis Functions for Efficient Function Learningcited by 4
  2. 2026Making Gaussian Kolmogorov-Arnold Networks Reliable and Accuratecited by 3
  3. 2026Partition-of-Unity Gaussian Kolmogorov-Arnold Networkscited by 1
  4. 2025SUPN: Shallow Universal Polynomial Networks for Efficient Function Approximationcited by 0