# Activation Space Selectable Kolmogorov-Arnold Networks

arXiv: https://arxiv.org/abs/2408.08338 · HTML: https://mlsanity.com/paper/2408.08338/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 15 Aug 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [Deep Learning in Spiking Neural Networks](https://mlsanity.com/paper/1804.08150/) (2018)
- [Recurrent Neural Network Regularization](https://mlsanity.com/paper/1409.2329/) (2014)
- [Wav-KAN: Wavelet Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.12832/) (2024)
- [S2-MLP: Spatial-Shift MLP Architecture for Vision](https://mlsanity.com/paper/2106.07477/) (2021)
- [Wavelet Convolutions for Large Receptive Fields](https://mlsanity.com/paper/2407.05848/) (2024)
- [Wavelet Integrated CNNs for Noise-Robust Image Classification](https://mlsanity.com/paper/2005.03337/) (2020)
- [Text Matching as Image Recognition](https://mlsanity.com/paper/1602.06359/) (2016)
- [KAGNNs: Kolmogorov-Arnold Networks meet Graph Learning](https://mlsanity.com/paper/2406.18380/) (2024)
- [BSRBF-KAN: A combination of B-splines and Radial Basis Functions in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.11173/) (2024)
- [Recurrent Attentional Reinforcement Learning for Multi-label Image Recognition](https://mlsanity.com/paper/1712.07465/) (2017)
- [Predicting the Future of AI with AI: High-quality link prediction in an exponentially growing knowledge network](https://mlsanity.com/paper/2210.00881/) (2022)
- [ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLU](https://mlsanity.com/paper/2406.02075/) (2024)
- [WaveCNet: Wavelet Integrated CNNs to Suppress Aliasing Effect for Noise-Robust Image Classification](https://mlsanity.com/paper/2107.13335/) (2021)
- [MLP Can Be A Good Transformer Learner](https://mlsanity.com/paper/2404.05657/) (2024)

## Built on by (most cited first, 9 of 9)

- [A Survey on Kolmogorov-Arnold Network](https://mlsanity.com/paper/2411.06078/) (2024)
- [FC-KAN: Function Combinations in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2409.01763/) (2024)
- [MedKAN: An Advanced Kolmogorov-Arnold Network for Medical Image Classification](https://mlsanity.com/paper/2502.18416/) (2025)
- [Kolmogorov-Arnold Networks in Low-Data Regimes: A Comparative Study with Multilayer Perceptrons](https://mlsanity.com/paper/2409.10463/) (2024)
- [AF-KAN: Activation Function-Based Kolmogorov-Arnold Networks for Efficient Representation Learning](https://mlsanity.com/paper/2503.06112/) (2025)
- [Vision KAN: Towards an Attention-Free Backbone for Vision with Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2601.21541/) (2026)
- [Capsule-ConvKAN: A Hybrid Neural Approach to Medical Image Classification](https://mlsanity.com/paper/2507.06417/) (2025)
- [Combinations of Fast Activation and Trigonometric Functions in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2508.11876/) (2025)
- [Enhanced Feature Extraction for IoT Network Intrusion Detection Using GNNs and KAN](https://mlsanity.com/paper/2607.02981/) (2026)

## Abstract (arXiv)

The multilayer perceptron (MLP), a fundamental paradigm in current artificial intelligence, is widely applied in fields such as computer vision and natural language processing. However, the recently proposed Kolmogorov-Arnold Network (KAN), based on nonlinear additive connections, has been proven to achieve performance comparable to MLPs with significantly fewer parameters. Despite this potential, the use of a single activation function space results in reduced performance of KAN and related works across different tasks. To address this issue, we propose an activation space Selectable KAN (S-KAN). S-KAN employs an adaptive strategy to choose the possible activation mode for data at each feedforward KAN node. Our approach outperforms baseline methods in seven representative function fitting tasks and significantly surpasses MLP methods with the same level of parameters. Furthermore, we extend the structure of S-KAN and propose an activation space selectable Convolutional KAN (S-ConvKAN), which achieves leading results on four general image classification datasets. Our method mitigates the performance variability of the original KAN across different tasks and demonstrates through extensive experiments that feedforward KANs with selectable activations can achieve or even exceed the performance of MLP-based methods. This work contributes to the understanding of the data-centric design of new AI paradigms and provides a foundational reference for innovations in KAN-based network architectures.