# A Gated Residual Kolmogorov-Arnold Networks for Mixtures of Experts

arXiv: https://arxiv.org/abs/2409.15161 · HTML: https://mlsanity.com/paper/2409.15161/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Builds on (most cited first, 20 of 20)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [N-BEATS: Neural basis expansion analysis for interpretable time series forecasting](https://mlsanity.com/paper/1905.10437/) (2019)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Expert Gate: Lifelong Learning with a Network of Experts](https://mlsanity.com/paper/1611.06194/) (2016)
- [Wav-KAN: Wavelet Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.12832/) (2024)
- [Kolmogorov-Arnold Networks (KANs) for Time Series Analysis](https://mlsanity.com/paper/2405.08790/) (2024)
- [TKAN: Temporal Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2405.07344/) (2024)
- [Kolmogorov-Arnold Networks for Time Series: Bridging Predictive Power and Interpretability](https://mlsanity.com/paper/2406.02496/) (2024)
- [Network of Experts for Large-Scale Image Categorization](https://mlsanity.com/paper/1604.06119/) (2016)
- [A Temporal Kolmogorov-Arnold Transformer for Time Series Forecasting](https://mlsanity.com/paper/2406.02486/) (2024)
- [Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2102.00397/) (2021)
- [SigKAN: Signature-Weighted Kolmogorov-Arnold Networks for Time Series](https://mlsanity.com/paper/2406.17890/) (2024)
- [Deep State Space Recurrent Neural Networks for Time Series Forecasting](https://mlsanity.com/paper/2407.15236/) (2024)

## Built on by (most cited first, 6 of 6)

- [ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inference](https://mlsanity.com/paper/2510.26730/) (2025)
- [A Temporal Linear Network for Time Series Forecasting](https://mlsanity.com/paper/2410.21448/) (2024)
- [Deep Learning for VWAP Execution in Crypto Markets: Beyond the Volume Curve](https://mlsanity.com/paper/2502.13722/) (2025)
- [Recurrent Neural Networks for Dynamic VWAP Execution: Adaptive Trading Strategies with Temporal Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2502.18177/) (2025)
- [VWAP Execution with Signature-Enhanced Transformers: A Multi-Asset Learning Approach](https://mlsanity.com/paper/2503.02680/) (2025)
- [Temporal Functional Circuits: From Spline Plots to Faithful Explanations in KAN Forecasting](https://mlsanity.com/paper/2605.05685/) (2026)

## Abstract (arXiv)

This paper introduces KAMoE, a novel Mixture of Experts (MoE) framework based on Gated Residual Kolmogorov-Arnold Networks (GRKAN). We propose GRKAN as an alternative to the traditional gating function, aiming to enhance efficiency and interpretability in MoE modeling. Through extensive experiments on digital asset markets and real estate valuation, we demonstrate that KAMoE consistently outperforms traditional MoE architectures across various tasks and model types. Our results show that GRKAN exhibits superior performance compared to standard Gating Residual Networks, particularly in LSTM-based models for sequential tasks. We also provide insights into the trade-offs between model complexity and performance gains in MoE and KAMoE architectures.