# Kolmogorov-Arnold Networks in Low-Data Regimes: A Comparative Study with Multilayer Perceptrons

arXiv: https://arxiv.org/abs/2409.10463 · HTML: https://mlsanity.com/paper/2409.10463/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 16 Sep 2024.

## Builds on (most cited first, 21 of 21)

- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2109.14545/) (2021)
- [Wav-KAN: Wavelet Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.12832/) (2024)
- [A comprehensive and FAIR comparison between MLP and KAN representations for differential equations and operator networks](https://mlsanity.com/paper/2406.02917/) (2024)
- [On the Impact of the Activation Function on Deep Neural Networks Training](https://mlsanity.com/paper/1902.06853/) (2019)
- [A survey on modern trainable activation functions](https://mlsanity.com/paper/2005.00817/) (2020)
- [The Kolmogorov-Arnold representation theorem revisited](https://mlsanity.com/paper/2007.15884/) (2020)
- [Learning Activation Functions to Improve Deep Neural Networks](https://mlsanity.com/paper/1412.6830/) (2014)
- [Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods](https://mlsanity.com/paper/1903.11260/) (2019)
- [A representer theorem for deep neural networks](https://mlsanity.com/paper/1802.09210/) (2018)
- [Parametric Exponential Linear Unit for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1605.09332/) (2016)
- [How important are activation functions in regression and classification? A survey, performance comparison, and future directions](https://mlsanity.com/paper/2209.02681/) (2022)
- [A Benchmarking Study of Kolmogorov-Arnold Networks on Tabular Data](https://mlsanity.com/paper/2406.14529/) (2024)
- [How to Learn More? Exploring Kolmogorov-Arnold Networks for Hyperspectral Image Classification](https://mlsanity.com/paper/2406.15719/) (2024)
- [Activation Space Selectable Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2408.08338/) (2024)
- [Weighted Sigmoid Gate Unit for an Activation Function of Deep Neural Network](https://mlsanity.com/paper/1810.01829/) (2018)
- [ErfAct and Pserf: Non-monotonic Smooth Trainable Activation Functions](https://mlsanity.com/paper/2109.04386/) (2021)
- [Stochastic Adaptive Activation Function](https://mlsanity.com/paper/2210.11672/) (2022)
- [Adaptive Activation Functions for Predictive Modeling with Sparse Experimental Data](https://mlsanity.com/paper/2402.05401/) (2024)

## Built on by (most cited first, 5 of 5)

- [Multifidelity Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.14764/) (2024)
- [Kolmogorov-Arnold Network Autoencoders](https://mlsanity.com/paper/2410.02077/) (2024)
- [KANs for Computer Vision: An Experimental Study](https://mlsanity.com/paper/2411.18224/) (2024)
- [Efficiency Bottlenecks of Convolutional Kolmogorov-Arnold Networks: A Comprehensive Scrutiny with ImageNet, AlexNet, LeNet and Tabular Classification](https://mlsanity.com/paper/2501.15757/) (2025)
- [Process-Informed Forecasting of Complex Thermal Dynamics in Pharmaceutical Manufacturing](https://mlsanity.com/paper/2509.20349/) (2025)

## Abstract (arXiv)

Multilayer Perceptrons (MLPs) have long been a cornerstone in deep learning, known for their capacity to model complex relationships. Recently, Kolmogorov-Arnold Networks (KANs) have emerged as a compelling alternative, utilizing highly flexible learnable activation functions directly on network edges, a departure from the neuron-centric approach of MLPs. However, KANs significantly increase the number of learnable parameters, raising concerns about their effectiveness in data-scarce environments. This paper presents a comprehensive comparative study of MLPs and KANs from both algorithmic and experimental perspectives, with a focus on low-data regimes. We introduce an effective technique for designing MLPs with unique, parameterized activation functions for each neuron, enabling a more balanced comparison with KANs. Using empirical evaluations on simulated data and two real-world data sets from medicine and engineering, we explore the trade-offs between model complexity and accuracy, with particular attention to the role of network depth. Our findings show that MLPs with individualized activation functions achieve significantly higher predictive accuracy with only a modest increase in parameters, especially when the sample size is limited to around one hundred. For example, in a three-class classification problem within additive manufacturing, MLPs achieve a median accuracy of 0.91, significantly outperforming KANs, which only reach a median accuracy of 0.53 with default hyperparameters. These results offer valuable insights into the impact of activation function selection in neural networks.