mlsanityData to 7 Oct 2026

A Gated Residual Kolmogorov-Arnold Networks for Mixtures of Experts

arXiv 2409.15161 ↗First posted 23 Sep 2024cs.LG cs.NEcited by 6

On the map: Mixture-of-experts → MoE theory and gating functions

Abstract (arXiv)

This paper introduces KAMoE, a novel Mixture of Experts (MoE) framework based on Gated Residual Kolmogorov-Arnold Networks (GRKAN). We propose GRKAN as an alternative to the traditional gating function, aiming to enhance efficiency and interpretability in MoE modeling. Through extensive experiments on digital asset markets and real estate valuation, we demonstrate that KAMoE consistently outperforms traditional MoE architectures across various tasks and model types. Our results show that GRKAN exhibits superior performance compared to standard Gating Residual Networks, particularly in LSTM-based models for sequential tasks. We also provide insights into the trade-offs between model complexity and performance gains in MoE and KAMoE architectures.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts. Builds on 20 of its 26 references that are in the map. First posted on arXiv 23 Sep 2024.

Builds on

20 of 26 references.

References · 20

  1. 2016Layer Normalizationcited by 5,579
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  4. 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,004
  5. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  6. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  7. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  8. 2019N-BEATS: Neural basis expansion analysis for interpretable time series forecastingcited by 744
  9. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  10. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  11. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 369
  12. 2024Wav-KAN: Wavelet Kolmogorov-Arnold Networkscited by 118
8 more
  1. 2024Kolmogorov-Arnold Networks (KANs) for Time Series Analysiscited by 97
  2. 2024TKAN: Temporal Kolmogorov-Arnold Networkscited by 70
  3. 2024Kolmogorov-Arnold Networks for Time Series: Bridging Predictive Power and Interpretabilitycited by 69
  4. 2016Network of Experts for Large-Scale Image Categorizationcited by 54
  5. 2024A Temporal Kolmogorov-Arnold Transformer for Time Series Forecastingcited by 50
  6. 2021Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecastingcited by 29
  7. 2024SigKAN: Signature-Weighted Kolmogorov-Arnold Networks for Time Seriescited by 13
  8. 2024Deep State Space Recurrent Neural Networks for Time Series Forecastingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 4; 2026: 1.

6 papers.

  1. 2025ExpertFlow: Adaptive Expert Scheduling and Memory Coordination for Efficient MoE Inferencecited by 5
  2. 2024A Temporal Linear Network for Time Series Forecastingcited by 4
  3. 2025Deep Learning for VWAP Execution in Crypto Markets: Beyond the Volume Curvecited by 4
  4. 2025Recurrent Neural Networks for Dynamic VWAP Execution: Adaptive Trading Strategies with Temporal Kolmogorov-Arnold Networkscited by 2
  5. 2025VWAP Execution with Signature-Enhanced Transformers: A Multi-Asset Learning Approachcited by 1
  6. 2026Temporal Functional Circuits: From Spline Plots to Faithful Explanations in KAN Forecastingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functions
  2. 2023Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts
  3. 2026TradingMoE: Routing the Right Experts in Evolving Markets
  4. 2026∞-MoE: Generalizing Mixture of Experts to Infinite Experts
  5. 2025LLM-Based Routing in Mixture of Experts: A Novel Framework for Trading
  6. 2023Improving Expert Specialization in Mixture of Experts
  7. 2025Rethinking Sparse Mixture of Experts from a Unified Perspective
  8. 2025Finding Fantastic Experts in MoEs: A Unified Study for Expert Dropping Strategies and Observations