mlsanityData to 7 Oct 2026

AF-KAN: Activation Function-Based Kolmogorov-Arnold Networks for Efficient Representation Learning

arXiv 2503.06112 ↗First posted 8 Mar 2025cs.LG cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Kolmogorov-Arnold Networks (KANs) have inspired numerous works exploring their applications across a wide range of scientific problems, with the potential to replace Multilayer Perceptrons (MLPs). While many KANs are designed using basis and polynomial functions, such as B-splines, ReLU-KAN utilizes a combination of ReLU functions to mimic the structure of B-splines and take advantage of ReLU's speed. However, ReLU-KAN is not built for multiple inputs, and its limitations stem from ReLU's handling of negative values, which can restrict feature extraction. To address these issues, we introduce Activation Function-Based Kolmogorov-Arnold Networks (AF-KAN), expanding ReLU-KAN with various activations and their function combinations. This novel KAN also incorporates parameter reduction methods, primarily attention mechanisms and data normalization, to enhance performance on image classification datasets. We explore different activation functions, function combinations, grid sizes, and spline orders to validate the effectiveness of AF-KAN and determine its optimal configuration. In the experiments, AF-KAN significantly outperforms MLP, ReLU-KAN, and other KANs with the same parameter count. It also remains competitive even when using fewer than 6 to 10 times the parameters while maintaining the same network structure. However, AF-KAN requires a longer training time and consumes more FLOPs. The repository for this work is available at https://github.com/hoangthangta/All-KAN.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 55 of its 70 references that are in the map. First posted on arXiv 8 Mar 2025.

Builds on

55 of 70 references.

References · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  4. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  5. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  6. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  7. 2017Self-Normalizing Neural Networkscited by 591
  8. 2015Empirical Evaluation of Rectified Activations in Convolutional Networkcited by 515
  9. 2024U-KAN Makes Strong Backbone for Medical Image Segmentation and Generationcited by 176
  10. 2024Kolmogorov-Arnold Networks are Radial Basis Function Networkscited by 125
  11. 2024Wav-KAN: Wavelet Kolmogorov-Arnold Networkscited by 118
  12. 2024KAN 2.0: Kolmogorov-Arnold Networks Meet Sciencecited by 116
38 more
  1. 2024Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximationcited by 115
  2. 2024Kolmogorov-Arnold Networks (KANs) for Time Series Analysiscited by 97
  3. 2020The Kolmogorov-Arnold representation theorem revisitedcited by 89
  4. 2024DeepOKAN: Deep Operator Network Based on Kolmogorov Arnold Networks for Mechanics Problemscited by 82
  5. 2024Convolutional Kolmogorov-Arnold Networkscited by 76
  6. 2024KAN or MLP: A Fairer Comparisoncited by 71
  7. 2024TKAN: Temporal Kolmogorov-Arnold Networkscited by 70
  8. 2024Kolmogorov-Arnold Networks for Time Series: Bridging Predictive Power and Interpretabilitycited by 69
  9. 2024A Survey on Kolmogorov-Arnold Networkcited by 67
  10. 2024fKAN: Fractional Kolmogorov-Arnold Networks with trainable Jacobi basis functionscited by 60
  11. 2024KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physicscited by 58
  12. 2024Kolmogorov-Arnold Transformercited by 56
  13. 2024A Temporal Kolmogorov-Arnold Transformer for Time Series Forecastingcited by 50
  14. 2024KAGNNs: Kolmogorov-Arnold Networks meet Graph Learningcited by 48
  15. 2024BSRBF-KAN: A combination of B-splines and Radial Basis Functions in Kolmogorov-Arnold Networkscited by 43
  16. 2024Enhancing Graph Collaborative Filtering with FourierKAN Feature Transformationcited by 41
  17. 2024Demonstrating the Efficacy of Kolmogorov-Arnold Networks in Vision Taskscited by 39
  18. 2024KANQAS: Kolmogorov-Arnold Network for Quantum Architecture Searchcited by 36
  19. 2024Exploring the Potential of Polynomial Basis Functions in Kolmogorov-Arnold Networks: A Comparative Study of Different Groups of Polynomialscited by 33
  20. 2024Unveiling the Power of Wavelets: A Wavelet-based Kolmogorov-Arnold Network for Hyperspectral Image Classificationcited by 33
  21. 2024ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLUcited by 32
  22. 2024GraphKAN: Enhancing Feature Extraction with Graph Kolmogorov Arnold Networkscited by 31
  23. 2024rKAN: Rational Kolmogorov-Arnold Networkscited by 31
  24. 2024Kolmogorov-Arnold Graph Neural Networkscited by 31
  25. 2024Exploring the Limitations of Kolmogorov-Arnold Networks in Classification: Insights to Software Training and Hardware Implementationcited by 22
  26. 2024CoxKAN: Kolmogorov-Arnold Networks for Interpretable, High-Performance Survival Analysiscited by 22
  27. 2024FC-KAN: Function Combinations in Kolmogorov-Arnold Networkscited by 18
  28. 2024Physics-Informed Kolmogorov-Arnold Networks for Power System Dynamicscited by 14
  29. 2024Higher-order-ReLU-KANs (HRKANs) for solving physics-informed neural networks (PINNs) more accurately, robustly and fastercited by 13
  30. 2024Kolmogorov-Arnold Network for Online Reinforcement Learningcited by 9
  31. 2024Activation Space Selectable Kolmogorov-Arnold Networkscited by 9
  32. 2024Kolmogorov-Arnold Network Autoencoderscited by 8
  33. 2024A First Look at Kolmogorov-Arnold Networks in Surrogate-assisted Evolutionary Algorithmscited by 7
  34. 2025Kolmogorov-Arnold Recurrent Network for Short Term Load Forecasting Across Diverse Consumerscited by 7
  35. 2025PRKAN: Parameter-Reduced Kolmogorov-Arnold Networkscited by 7
  36. 2024Architectural Scaling Surpass Basis Complexity? Efficient KANs with Single-Parameter Designcited by 6
  37. 2024Sparks of Quantum Advantage and Rapid Retraining in Machine Learningcited by 5
  38. 2024Effective Integration of KAN for Keyword Spottingcited by 5

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Concurrent training methods for Kolmogorov-Arnold networks: Disjoint datasets and FPGA implementationcited by 2
  2. 2026RecKAN: Kolmogorov-Arnold Networks with a Learnable Recursive Polynomial Basiscited by 1
  3. 2026Sample-Efficiency of Kolmogorov-Arnold Networkscited by 0