mlsanityData to 7 Oct 2026

ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU

arXiv 2406.02075 ↗First posted 4 Jun 2024cs.LG cs.NEcited by 32

Not on the map. Why?

Abstract (arXiv)

Limited by the complexity of basis function (B-spline) calculations, Kolmogorov-Arnold Networks (KAN) suffer from restricted parallel computing capability on GPUs. This paper proposes a novel ReLU-KAN implementation that inherits the core idea of KAN. By adopting ReLU (Rectified Linear Unit) and point-wise multiplication, we simplify the design of KAN's basis function and optimize the computation process for efficient CUDA computing. The proposed ReLU-KAN architecture can be readily implemented on existing deep learning frameworks (e.g., PyTorch) for both inference and training. Experimental results demonstrate that ReLU-KAN achieves a 20x speedup compared to traditional KAN with 4-layer networks. Furthermore, ReLU-KAN exhibits a more stable training process with superior fitting ability while preserving the "catastrophic forgetting avoidance" property of KAN. You can get the code in https://github.com/quiqi/relu_kan

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 4 of its 7 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

4 of 7 references.

References · 4

  1. 2024Wav-KAN: Wavelet Kolmogorov-Arnold Networkscited by 118
  2. 2024Kolmogorov-Arnold Networks (KANs) for Time Series Analysiscited by 97
  3. 2024DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problemscited by 82
  4. 2024TKAN: Temporal Kolmogorov-Arnold Networkscited by 70

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 10; 2026: 8.

32 papers.

Sort:

By area:

  1. 2024Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networkscited by 37
  2. 2024Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problemscited by 35
  3. 2024FC-KAN: Function Combinations in Kolmogorov-Arnold Networkscited by 18
  4. 2024Reduced Effectiveness of Kolmogorov-Arnold Networks on Functions with Noisecited by 17
  5. 2024Higher-order-ReLU-KANs (HRKANs) for solving physics-informed neural networks (PINNs) more accurately, robustly and fastercited by 13
  6. 2024Activation Space Selectable Kolmogorov-Arnold Networkscited by 9
  7. 2024Multifidelity Kolmogorov-Arnold Networkscited by 9
  8. 2024KAN versus MLP on Irregular or Noisy Functionscited by 7
  9. 2026FEKAN: Feature-Enriched Kolmogorov-Arnold Networkscited by 5
  10. 2025Introducing the Short-Time Fourier Kolmogorov Arnold Network: A Dynamic Graph CNN Approach for Tree Species Classification in 3D Point Cloudscited by 4
  11. 2025ChemKANs for Combustion Chemistry Modeling and Accelerationcited by 4
  12. 2025Leveraging KANs for Expedient Training of Multichannel MLPs via Preconditioning and Geometric Refinementcited by 4
20 more
  1. 2024KAN v.s. MLP for Offline Reinforcement Learningcited by 3
  2. 2024Evaluating Federated Kolmogorov-Arnold Networks on Non-IID Datacited by 3
  3. 2025AF-KAN: Activation Function-Based Kolmogorov-Arnold Networks for Efficient Representation Learningcited by 3
  4. 2025Automatic Grid Updates for Kolmogorov-Arnold Networks using Layer Histogramscited by 3
  5. 2026Making Gaussian Kolmogorov-Arnold Networks Reliable and Accuratecited by 3
  6. 2024Kolmogorov Arnold Networks in Fraud Detection: Bridging the Gap Between Theory and Practicecited by 2
  7. 2024P1-KAN: an effective Kolmogorov-Arnold network with application to hydraulic valley optimizationcited by 2
  8. 2024Uncertainty Quantification with Bayesian Higher Order ReLU KANscited by 1
  9. 2025Scaled Supervision is an Implicit Lipschitz Regularizercited by 1
  10. 2025Sinusoidal Approximation Theorem for Kolmogorov-Arnold Networkscited by 1
  11. 2026Multilevel Training for Kolmogorov Arnold Networkscited by 1
  12. 2026Partition-of-Unity Gaussian Kolmogorov-Arnold Networkscited by 1
  13. 2026SechKAN: Kolmogorov-Arnold Networks with Hyperbolic Secant Functionscited by 1
  14. 2026RecKAN: Kolmogorov-Arnold Networks with a Learnable Recursive Polynomial Basiscited by 1
  15. 2024On the Convergence of (Stochastic) Gradient Descent for Kolmogorov--Arnold Networkscited by 0
  16. 2025HKAN: Hierarchical Kolmogorov-Arnold Network without Backpropagationcited by 0
  17. 2025Combinations of Fast Activation and Trigonometric Functions in Kolmogorov-Arnold Networkscited by 0
  18. 2025PolyKAN: A Polyhedral Analysis Framework for Provable and Approximately Optimal KAN Compressioncited by 0
  19. 2026A Novel Tensor Product-Based Neural Network for Solving Partial Differential Equationscited by 0
  20. 2026Evaluating the Effectiveness of SechKAN on 1D Datacited by 0