# Improving Memory Efficiency for Training KANs via Meta Learning

arXiv: https://arxiv.org/abs/2506.07549 · HTML: https://mlsanity.com/paper/2506.07549/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [What Can Transformers Learn In-Context? A Case Study of Simple Function Classes](https://mlsanity.com/paper/2208.01066/) (2022)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks](https://mlsanity.com/paper/2106.04489/) (2021)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing](https://mlsanity.com/paper/2111.15666/) (2021)
- [U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation](https://mlsanity.com/paper/2406.02918/) (2024)
- [Kolmogorov-Arnold Networks are Radial Basis Function Networks](https://mlsanity.com/paper/2405.06721/) (2024)
- [Learning the Pareto Front with Hypernetworks](https://mlsanity.com/paper/2010.04104/) (2020)
- [Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation](https://mlsanity.com/paper/2405.07200/) (2024)
- [Tackling the Curse of Dimensionality with Physics-Informed Neural Networks](https://mlsanity.com/paper/2307.12306/) (2023)
- [Convolutional Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.13155/) (2024)
- [KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physics](https://mlsanity.com/paper/2407.04192/) (2024)
- [Kolmogorov-Arnold Transformer](https://mlsanity.com/paper/2409.10594/) (2024)
- [KAGNNs: Kolmogorov-Arnold Networks meet Graph Learning](https://mlsanity.com/paper/2406.18380/) (2024)
- [Meta-learning for mixed linear regression](https://mlsanity.com/paper/2002.08936/) (2020)
- [On the expressiveness and spectral bias of KANs](https://mlsanity.com/paper/2410.01803/) (2024)
- [Learning Implicitly Recurrent CNNs Through Parameter Sharing](https://mlsanity.com/paper/1902.09701/) (2019)
- [Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies](https://mlsanity.com/paper/2407.01092/) (2024)
- [Small Sample Learning in Big Data Era](https://mlsanity.com/paper/1808.04572/) (2018)
- [Generalization Bounds and Model Complexity for Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2410.08026/) (2024)
- [Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks](https://mlsanity.com/paper/2107.02378/) (2021)
- [Finding Local Diffusion Schr"odinger Bridge using Kolmogorov-Arnold Network](https://mlsanity.com/paper/2502.19754/) (2025)
- [KAC: Kolmogorov-Arnold Classifier for Continual Learning](https://mlsanity.com/paper/2503.21076/) (2025)

## Built on by (most cited first, 3 of 3)

- [MetaCluster: Enabling Deep Compression of Kolmogorov-Arnold Network](https://mlsanity.com/paper/2510.19105/) (2025)
- [TruKAN: Towards More Efficient Kolmogorov-Arnold Networks Using Truncated Power Functions](https://mlsanity.com/paper/2602.03879/) (2026)
- [Structural Kolmogorov-Arnold Convolutions: Learnable Function on the Values or the Filter Shape as Parameter-Efficient Alternative to Per-Edge Convolutional KANs](https://mlsanity.com/paper/2606.24371/) (2026)

## Abstract (arXiv)

Inspired by the Kolmogorov-Arnold representation theorem, KANs offer a novel framework for function approximation by replacing traditional neural network weights with learnable univariate functions. This design demonstrates significant potential as an efficient and interpretable alternative to traditional MLPs. However, KANs are characterized by a substantially larger number of trainable parameters, leading to challenges in memory efficiency and higher training costs compared to MLPs. To address this limitation, we propose to generate weights for KANs via a smaller meta-learner, called MetaKANs. By training KANs and MetaKANs in an end-to-end differentiable manner, MetaKANs achieve comparable or even superior performance while significantly reducing the number of trainable parameters and maintaining promising interpretability. Extensive experiments on diverse benchmark tasks, including symbolic regression, partial differential equation solving, and image classification, demonstrate the effectiveness of MetaKANs in improving parameter efficiency and memory usage. The proposed method provides an alternative technique for training KANs, that allows for greater scalability and extensibility, and narrows the training cost gap with MLPs stated in the original paper of KANs. Our code is available at https://github.com/Murphyzc/MetaKAN.