# Wav-KAN: Wavelet Kolmogorov-Arnold Networks

arXiv: https://arxiv.org/abs/2405.12832 · HTML: https://mlsanity.com/paper/2405.12832/

## Key facts

- Cited by 122 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 21 May 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [A Survey on Neural Network Interpretability](https://mlsanity.com/paper/2012.14261/) (2020)
- [The Alignment Problem from a Deep Learning Perspective](https://mlsanity.com/paper/2209.00626/) (2022)
- [An Overview of Catastrophic AI Risks](https://mlsanity.com/paper/2306.12001/) (2023)
- [WIRE: Wavelet Implicit Neural Representations](https://mlsanity.com/paper/2301.05187/) (2023)
- [On Interpretability of Artificial Neural Networks: A Survey](https://mlsanity.com/paper/2001.02522/) (2020)
- [ExSpliNet: An interpretable and expressive spline-based neural network](https://mlsanity.com/paper/2205.01510/) (2022)

## Built on by (most cited first, 30 of 122)

- [KAN 2.0: Kolmogorov-Arnold Networks Meet Science](https://mlsanity.com/paper/2408.10205/) (2024)
- [A Survey on Kolmogorov-Arnold Network](https://mlsanity.com/paper/2411.06078/) (2024)
- [Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks](https://mlsanity.com/paper/2406.11045/) (2024)
- [fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functions](https://mlsanity.com/paper/2406.07456/) (2024)
- [A Temporal Kolmogorov-Arnold Transformer for Time Series Forecasting](https://mlsanity.com/paper/2406.02486/) (2024)
- [Kolmogorov-Arnold Network for Satellite Image Classification in Remote Sensing](https://mlsanity.com/paper/2406.00600/) (2024)
- [Adaptive Training of Grid-Dependent Physics-Informed Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2407.17611/) (2024)
- [KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Search](https://mlsanity.com/paper/2406.17630/) (2024)
- [On the expressiveness and spectral bias of KANs](https://mlsanity.com/paper/2410.01803/) (2024)
- [Exploring the Potential of Polynomial Basis Functions in Kolmogorov-Arnold Networks: A Comparative Study of Different Groups of Polynomials](https://mlsanity.com/paper/2406.02583/) (2024)
- [rKAN: Rational Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.14495/) (2024)
- [Unveiling the Power of Wavelets: A Wavelet-based Kolmogorov-Arnold Network for Hyperspectral Image Classification](https://mlsanity.com/paper/2406.07869/) (2024)
- [Suitability of KANs for Computer Vision: A preliminary investigation](https://mlsanity.com/paper/2406.09087/) (2024)
- [ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU](https://mlsanity.com/paper/2406.02075/) (2024)
- [Physics Informed Kolmogorov-Arnold Neural Networks for Dynamical Analysis via Efficent-KAN and WAV-KAN](https://mlsanity.com/paper/2407.18373/) (2024)
- [Exploring the Limitations of Kolmogorov-Arnold Networks in Classification: Insights to Software Training and Hardware Implementation](https://mlsanity.com/paper/2407.17790/) (2024)
- [FC-KAN: Function Combinations in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2409.01763/) (2024)
- [Reduced Effectiveness of Kolmogorov-Arnold Networks on Functions with Noise](https://mlsanity.com/paper/2407.14882/) (2024)
- [SineKAN: Kolmogorov-Arnold Networks Using Sinusoidal Activation Functions](https://mlsanity.com/paper/2407.04149/) (2024)
- [DropKAN: Regularizing KANs by masking post-activations](https://mlsanity.com/paper/2407.13044/) (2024)
- [SigKAN: Signature-Weighted Kolmogorov-Arnold Networks for Time Series](https://mlsanity.com/paper/2406.17890/) (2024)
- [Higher-order-ReLU-KANs (HRKANs) for solving physics-informed neural networks (PINNs) more accurately, robustly and faster](https://mlsanity.com/paper/2409.14248/) (2024)
- [SPIKANs: Separable Physics-Informed Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2411.06286/) (2024)
- [On the Robustness of Kolmogorov-Arnold Networks: An Adversarial Perspective](https://mlsanity.com/paper/2408.13809/) (2024)
- [Generalization Bounds and Model Complexity for Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.08026/) (2024)
- [Kolmogorov-Arnold Network for Online Reinforcement Learning](https://mlsanity.com/paper/2408.04841/) (2024)
- [Activation Space Selectable Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2408.08338/) (2024)
- [Addressing common misinterpretations of KART and UAT in neural network literature](https://mlsanity.com/paper/2408.16389/) (2024)
- [Data-driven model discovery with Kolmogorov-Arnold networks](https://mlsanity.com/paper/2409.15167/) (2024)
- [Multifidelity Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.14764/) (2024)

## Abstract (arXiv)

In this paper, we introduce Wav-KAN, an innovative neural network architecture that leverages the Wavelet Kolmogorov-Arnold Networks (Wav-KAN) framework to enhance interpretability and performance. Traditional multilayer perceptrons (MLPs) and even recent advancements like Spl-KAN face challenges related to interpretability, training speed, robustness, computational efficiency, and performance. Wav-KAN addresses these limitations by incorporating wavelet functions into the Kolmogorov-Arnold network structure, enabling the network to capture both high-frequency and low-frequency components of the input data efficiently. Wavelet-based approximations employ orthogonal or semi-orthogonal basis and maintain a balance between accurately representing the underlying data structure and avoiding overfitting to the noise. While continuous wavelet transform (CWT) has a lot of potentials, we also employed discrete wavelet transform (DWT) for multiresolution analysis, which obviated the need for recalculation of the previous steps in finding the details. Analogous to how water conforms to the shape of its container, Wav-KAN adapts to the data structure, resulting in enhanced accuracy, faster training speeds, and increased robustness compared to Spl-KAN and MLPs. Our results highlight the potential of Wav-KAN as a powerful tool for developing interpretable and high-performance neural networks, with applications spanning various fields. This work sets the stage for further exploration and implementation of Wav-KAN in frameworks such as PyTorch and TensorFlow, aiming to make wavelets in KAN as widespread as activation functions like ReLU and sigmoid in universal approximation theory (UAT). The codes to replicate the simulations are available at https://github.com/zavareh1/Wav-KAN.