# MetaCluster: Enabling Deep Compression of Kolmogorov-Arnold Network

arXiv: https://arxiv.org/abs/2510.19105 · HTML: https://mlsanity.com/paper/2510.19105/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 21 Oct 2025.

## Builds on (most cited first, 29 of 29)

- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [Extreme Compression of Large Language Models via Additive Quantization](https://mlsanity.com/paper/2401.06118/) (2024)
- [Compressing Deep Convolutional Networks using Vector Quantization](https://mlsanity.com/paper/1412.6115/) (2014)
- [U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation](https://mlsanity.com/paper/2406.02918/) (2024)
- [Using Fast Weights to Attend to the Recent Past](https://mlsanity.com/paper/1610.06258/) (2016)
- [Kolmogorov-Arnold Networks are Radial Basis Function Networks](https://mlsanity.com/paper/2405.06721/) (2024)
- [Learning the Pareto Front with Hypernetworks](https://mlsanity.com/paper/2010.04104/) (2020)
- [Wav-KAN: Wavelet Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.12832/) (2024)
- [Chebyshev Polynomial-Based Kolmogorov-Arnold Networks: An Efficient Architecture for Nonlinear Function Approximation](https://mlsanity.com/paper/2405.07200/) (2024)
- [Learning feed-forward one-shot learners](https://mlsanity.com/paper/1606.05233/) (2016)
- [GPTVQ: The Blessing of Dimensionality for LLM Quantization](https://mlsanity.com/paper/2402.15319/) (2024)
- [Convolutional Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.13155/) (2024)
- [KAN or MLP: A Fairer Comparison](https://mlsanity.com/paper/2407.16674/) (2024)
- [Kolmogorov Arnold Informed neural network: A physics-informed deep learning framework for solving forward and inverse problems based on Kolmogorov Arnold Networks](https://mlsanity.com/paper/2406.11045/) (2024)
- [KAN-ODEs: Kolmogorov-Arnold Network Ordinary Differential Equations for Learning Dynamical Systems and Hidden Physics](https://mlsanity.com/paper/2407.04192/) (2024)
- [VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2409.17066/) (2024)
- [Kolmogorov-Arnold Transformer](https://mlsanity.com/paper/2409.10594/) (2024)
- [Towards the Limit of Network Quantization](https://mlsanity.com/paper/1612.01543/) (2016)
- [rKAN: Rational Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2406.14495/) (2024)
- [DKM: Differentiable K-Means Clustering Layer for Neural Network Compression](https://mlsanity.com/paper/2108.12659/) (2021)
- [Learning Implicitly Recurrent CNNs Through Parameter Sharing](https://mlsanity.com/paper/1902.09701/) (2019)
- [Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies](https://mlsanity.com/paper/2407.01092/) (2024)
- [MatrixKAN: Parallelized Kolmogorov-Arnold Network](https://mlsanity.com/paper/2502.07176/) (2025)
- [FlashKAT: Understanding and Addressing Performance Bottlenecks in the Kolmogorov-Arnold Transformer](https://mlsanity.com/paper/2505.13813/) (2025)
- [Improving Memory Efficiency for Training KANs via Meta Learning](https://mlsanity.com/paper/2506.07549/) (2025)

## Built on by (most cited first, 2 of 2)

- [QuantKAN: A Unified Quantization Framework for Kolmogorov Arnold Networks](https://mlsanity.com/paper/2511.18689/) (2025)
- [SHARe-KAN: Post-Training Vector Quantization for Cache-Resident KAN Inference](https://mlsanity.com/paper/2512.15742/) (2025)

## Abstract (arXiv)

Kolmogorov-Arnold Networks (KANs) replace scalar weights with per-edge vectors of basis coefficients, thereby increasing expressivity and accuracy while also resulting in a multiplicative increase in parameters and memory. We propose MetaCluster, a framework that makes KANs highly compressible without sacrificing accuracy. Specifically, a lightweight meta-learner, trained jointly with the KAN, maps low-dimensional embeddings to coefficient vectors, thereby shaping them to lie on a low-dimensional manifold that is amenable to clustering. We then run K-means in coefficient space and replace per-edge vectors with shared centroids. Afterwards, the meta-learner can be discarded, and a brief fine-tuning of the centroid codebook recovers any residual accuracy loss. The resulting model stores only a small codebook and per-edge indices, exploiting the vector nature of KAN parameters to amortize storage across multiple coefficients. On MNIST, CIFAR-10, and CIFAR-100, across standard KANs and ConvKANs using multiple basis functions, MetaCluster achieves a reduction of up to 80× in parameter storage, with no loss in accuracy. Similarly, on high-dimensional equation modeling tasks, MetaCluster achieves a parameter reduction of 124.1×, without impacting performance. Code will be released upon publication.