# Kolmogorov GAM Networks are all you need!

arXiv: https://arxiv.org/abs/2501.00704 · HTML: https://mlsanity.com/paper/2501.00704/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 60 references that are in the map.
- First posted on arXiv 1 Jan 2025.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel](https://mlsanity.com/paper/1908.11775/) (2019)
- [Wav-KAN: Wavelet Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.12832/) (2024)
- [KAN 2.0: Kolmogorov-Arnold Networks Meet Science](https://mlsanity.com/paper/2408.10205/) (2024)
- [The Kolmogorov-Arnold representation theorem revisited](https://mlsanity.com/paper/2007.15884/) (2020)
- [Error bounds for deep ReLU networks using the Kolmogorov--Arnold superposition theorem](https://mlsanity.com/paper/1906.11945/) (2019)
- [Risk Bounds for High-dimensional Ridge Function Combinations Including Neural Networks](https://mlsanity.com/paper/1607.01434/) (2016)
- [Deep Learning: A Bayesian Perspective](https://mlsanity.com/paper/1706.00473/) (2017)
- [On the Kolmogorov neural networks](https://mlsanity.com/paper/2311.00049/) (2023)
- [FC-KAN: Function Combinations in Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2409.01763/) (2024)
- [SineKAN: Kolmogorov-Arnold Networks Using Sinusoidal Activation Functions](https://mlsanity.com/paper/2407.04149/) (2024)
- [A three layer neural network can represent any multivariate function](https://mlsanity.com/paper/2012.03016/) (2020)
- [Merging Two Cultures: Deep and Statistical Learning](https://mlsanity.com/paper/2110.11561/) (2021)
- [Hierarchical Neural Networks, p-Adic PDEs, and Applications to Image Processing](https://mlsanity.com/paper/2406.07790/) (2024)

## Built on by (most cited first, 1 of 1)

- [Horseshoe Mixtures-of-Experts (HS-MoE)](https://mlsanity.com/paper/2601.09043/) (2026)

## Abstract (arXiv)

Kolmogorov GAM (K-GAM) networks are shown to be an efficient architecture for training and inference. They are an additive model with an embedding that is independent of the function of interest. They provide an alternative to the transformer architecture. They are the machine learning version of Kolmogorov's Superposition Theorem (KST) which provides an efficient representations of a multivariate function. Such representations have use in machine learning for encoding dictionaries (a.k.a. "look-up" tables). KST theory also provides a representation based on translates of the K"oppen function. The goal of our paper is to interpret this representation in a machine learning context for applications in Artificial Intelligence (AI). Our architecture is equivalent to a topological embedding which is independent of the function together with an additive layer that uses a Generalized Additive Model (GAM). This provides a class of learning procedures with far fewer parameters than current deep learning algorithms. Implementation can be parallelizable which makes our algorithms computationally attractive. To illustrate our methodology, we use the Iris data from statistical learning. We also show that our additive model with non-linear embedding provides an alternative to transformer architectures which from a statistical viewpoint are kernel smoothers. Additive KAN models therefore provide a natural alternative to transformers. Finally, we conclude with directions for future research.