mlsanityData to 14 Oct 2026

Approximation Rates in Besov Norms and Sample-Complexity of Kolmogorov-Arnold Networks with Residual Connections

arXiv 2504.15110 ↗First posted 21 Apr 2025cs.LG cs.NA cs.NEcited by 4

Not on the map. Why?

Abstract (arXiv)

Inspired by the Kolmogorov-Arnold superposition theorem, Kolmogorov-Arnold Networks (KANs) have recently emerged as an improved backbone for most deep learning frameworks, promising more adaptivity than their multilayer perceptron (MLP) predecessor by allowing for trainable spline-based activation functions. In this paper, we probe the theoretical foundations of the KAN architecture by showing that it can optimally approximate any Besov function in Bsp,q(X) on a bounded open, or even fractal, domain X in ℝd at the optimal approximation rate with respect to any weaker Besov norm Bαp,q(X); where α < s. We complement our approximation result with a statistical guarantee by bounding the pseudodimension of the relevant class of Res-KANs. As an application of the latter, we directly deduce a dimension-free estimate on the sample complexity of a residual KAN model when learning a function of Besov regularity from N i.i.d. noiseless samples, showing that KANs can learn the smooth maps which they can approximate.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 26 of its 55 references that are in the map. First posted on arXiv 21 Apr 2025.

Builds on

26 of 55 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020GLU Variants Improve Transformercited by 1,804
  4. 2024KAN: Kolmogorov-Arnold Networkscited by 649
  5. 2019Universal Approximation with Deep Narrow Networkscited by 200
  6. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
  7. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 162
  8. 2020Minimum Width for Universal Approximationcited by 94
  9. 2021On the approximation of functions by tanh neural networkscited by 81
  10. 2018ResNet with one-neuron hidden layers is a Universal Approximatorcited by 79
  11. 2019Approximation spaces of deep neural networkscited by 51
  12. 2021Universal Approximation Theorems for Differentiable Geometric Deep Learningcited by 41
14 more
  1. 2021Deep Network Approximation: Achieving Arbitrary Accuracy with Fixed Number of Neuronscited by 41
  2. 2024Transformers are Minimax Optimal Nonparametric In-Context Learnerscited by 37
  3. 2024On the expressiveness and spectral bias of KANscited by 37
  4. 2021Elementary superexpressive activationscited by 36
  5. 2023Deep Network Approximation: Beyond ReLU to Diverse Activation Functionscited by 33
  6. 2022Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activationscited by 29
  7. 2022Designing Universal Causal Deep Learning Models: The Geometric (Hyper)Transformercited by 22
  8. 2022Neural Network Architecture Beyond Width and Depthcited by 17
  9. 2022Singular Value Perturbation and Deep Network Optimizationcited by 11
  10. 2021Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on H"older Classcited by 7
  11. 2024Don't Fear Peculiar Activation Functions: EUAF and Beyondcited by 7
  12. 2024Approximation Rates for Shallow ReLUk Neural Networks on Sobolev Spaces via the Radon Transformcited by 6
  13. 2024Scalable Message Passing Neural Networks: No Need for Attention in Large Graph Representation Learningcited by 4
  14. 2018An analysis of training and generalization errors in shallow and deep networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Necessary and sufficient conditions for universality of Kolmogorov-Arnold networkscited by 0
  2. 2026Posterior Contraction Rates for Sparse Kolmogorov-Arnold Networks in Anisotropic Besov Spacescited by 0
  3. 2026KAPLAN: Kolmogorov-Arnold Prognostic Learnable Activation Networks for Survival Analysiscited by 0
  4. 2026A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Practitioner's Guide to Kolmogorov-Arnold Networks
  2. 2025Sinusoidal Approximation Theorem for Kolmogorov-Arnold Networks
  3. 2026SparseKAN: Compressing Kolmogorov--Arnold Networks Across Basis Functions, Neurons, and Bits
  4. 2025GS-KAN: Parameter-Efficient Kolmogorov-Arnold Networks via Sprecher-Type Shared Basis Functions
  5. 2026KANtize: Exploring Low-bit Quantization of Kolmogorov-Arnold Networks for Efficient Inference
  6. 2026An Embedded RISC-V Evaluation of Kolmogorov--Arnold Networks in Hard-Constrained Recurrent Physics-Informed Models
  7. 2023On the Optimal Expressive Power of ReLU DNNs and Its Application in Approximation with Kolmogorov Superposition Theorem
  8. 2026Sample-Efficiency of Kolmogorov-Arnold Networks