# Higher-order-ReLU-KANs (HRKANs) for solving physics-informed neural networks (PINNs) more accurately, robustly and faster

arXiv: https://arxiv.org/abs/2409.14248 · HTML: https://mlsanity.com/paper/2409.14248/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 9 Aug 2024.

## Builds on (most cited first, 21 of 21)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019)
- [Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets](https://mlsanity.com/paper/2103.10974/) (2021)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [PDE-Net: Learning PDEs from Data](https://mlsanity.com/paper/1710.09668/) (2017)
- [Kolmogorov-Arnold Networks are Radial Basis Function Networks](https://mlsanity.com/paper/2405.06721/) (2024)
- [Wav-KAN: Wavelet Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.12832/) (2024)
- [Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation](https://mlsanity.com/paper/2405.07200/) (2024)
- [Physics-Informed Deep Neural Operator Networks](https://mlsanity.com/paper/2207.05748/) (2022)
- [DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problems](https://mlsanity.com/paper/2405.19143/) (2024)
- [Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks](https://mlsanity.com/paper/2406.11045/) (2024)
- [fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functions](https://mlsanity.com/paper/2406.07456/) (2024)
- [Optimal control of PDEs using physics-informed neural networks](https://mlsanity.com/paper/2111.09880/) (2021)
- [Implicit Neural Spatial Representations for Time-dependent PDEs](https://mlsanity.com/paper/2210.00124/) (2022)
- [rKAN: Rational Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.14495/) (2024)
- [Finite basis Kolmogorov-Arnold networks: domain decomposition for data-driven and physics-informed problems](https://mlsanity.com/paper/2406.19662/) (2024)
- [ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU](https://mlsanity.com/paper/2406.02075/) (2024)
- [RoPINN: Region Optimized Physics-Informed Neural Networks](https://mlsanity.com/paper/2405.14369/) (2024)
- [Physics-Informed Holomorphic Neural Networks (PIHNNs): Solving Linear Elasticity Problems](https://mlsanity.com/paper/2407.01088/) (2024)

## Built on by (most cited first, 13 of 13)

- [SPIKANs: Separable Physics-Informed Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2411.06286/) (2024)
- [Multifidelity Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.14764/) (2024)
- [FEKAN: Feature-Enriched Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2602.16530/) (2026)
- [Leveraging KANs for Expedient Training of Multichannel MLPs via Preconditioning and Geometric Refinement](https://mlsanity.com/paper/2505.18131/) (2025)
- [P1-KAN: an effective Kolmogorov-Arnold network with application to hydraulic valley optimization](https://mlsanity.com/paper/2410.03801/) (2024)
- [AF-KAN: Activation Function-Based Kolmogorov-Arnold Networks for Efficient Representation Learning](https://mlsanity.com/paper/2503.06112/) (2025)
- [Making Gaussian Kolmogorov-Arnold Networks Reliable and Accurate](https://mlsanity.com/paper/2604.21174/) (2026)
- [Uncertainty Quantification with Bayesian Higher Order ReLU KANs](https://mlsanity.com/paper/2410.01687/) (2024)
- [On the failure of ReLU activation for physics-informed machine learning](https://mlsanity.com/paper/2512.11184/) (2025)
- [A Dynamic Framework for Grid Adaptation in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2601.18672/) (2026)
- [Multilevel Training for Kolmogorov Arnold Networks](https://mlsanity.com/paper/2603.04827/) (2026)
- [Partition-of-Unity Gaussian Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2604.23599/) (2026)
- [A Novel Tensor Product-Based Neural Network for Solving Partial Differential Equations](https://mlsanity.com/paper/2605.29688/) (2026)

## Abstract (arXiv)

Finding solutions to partial differential equations (PDEs) is an important and essential component in many scientific and engineering discoveries. One of the common approaches empowered by deep learning is Physics-informed Neural Networks (PINNs). Recently, a new type of fundamental neural network model, Kolmogorov-Arnold Networks (KANs), has been proposed as a substitute of Multilayer Perceptions (MLPs), and possesses trainable activation functions. To enhance KANs in fitting accuracy, a modification of KANs, so called ReLU-KANs, using "square of ReLU" as the basis of its activation functions, has been suggested. In this work, we propose another basis of activation functions, namely, Higherorder-ReLU (HR), which is simpler than the basis of activation functions used in KANs, namely, Bsplines; allows efficient KAN matrix operations; and possesses smooth and non-zero higher-order derivatives, essential to physicsinformed neural networks. We name such KANs with Higher-order-ReLU (HR) as their activations, HRKANs. Our detailed experiments on two famous and representative PDEs, namely, the linear Poisson equation and nonlinear Burgers' equation with viscosity, reveal that our proposed Higher-order-ReLU-KANs (HRKANs) achieve the highest fitting accuracy and training robustness and lowest training time significantly among KANs, ReLU-KANs and HRKANs. The codes to replicate our experiments are available at https://github.com/kelvinhkcs/HRKAN.