mlsanityData to 7 Oct 2026

Mean-field limit from general mixtures of experts to quantum neural networks

arXiv 2501.14660 ↗First posted 24 Jan 2025math-ph cs.LG math.MPcited by 0

On the map: Mixture-of-experts → MoE theory and gating functions

Abstract (arXiv)

In this work, we study the asymptotic behavior of Mixture of Experts (MoE) trained via gradient flow on supervised learning problems. Our main result establishes the propagation of chaos for a MoE as the number of experts diverges. We demonstrate that the corresponding empirical measure of their parameters is close to a probability measure that solves a nonlinear continuity equation, and we provide an explicit convergence rate that depends solely on the number of experts. We apply our results to a MoE generated by a quantum neural network.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts. Builds on 15 of its 51 references that are in the map. First posted on arXiv 24 Jan 2025.

Builds on

15 of 51 references.

References · 15

  1. 2018Computational Optimal Transportcited by 765
  2. 2018Supervised learning with quantum enhanced feature spacescited by 654
  3. 2020The effect of data encoding on the expressive power of variational quantum machine learning modelscited by 328
  4. 2016Quantum Machine Learningcited by 325
  5. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 275
  6. 2020A rigorous and robust quantum speed-up in supervised machine learningcited by 270
  7. 2015Learning with a Wasserstein Losscited by 194
  8. 2019Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limitcited by 187
  9. 2018Trainability and Accuracy of Neural Networks: An Interacting Particle System Approachcited by 105
  10. 2020A Rigorous Framework for the Mean Field Limit of Multilayer Neural Networkscited by 68
  11. 2020A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depthcited by 56
  12. 2019Mean Field Analysis of Deep Neural Networkscited by 42
3 more
  1. 2019Wasserstein Distance based Deep Adversarial Transfer Learning for Intelligent Fault Diagnosiscited by 13
  2. 2021Learning quantum data with the quantum Earth Mover's distancecited by 13
  3. 2024Trained quantum neural networks are Gaussian processescited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025QMoE: A Quantum Mixture of Experts Framework for Scalable Quantum Neural Networks
  2. 2022Probabilistic partition of unity networks for high-dimensional regression problems
  3. 2026∞-MoE: Generalizing Mixture of Experts to Infinite Experts
  4. 2022Parameter-Efficient Mixture-of-Experts Architecture for Pre-trained Language Models
  5. 2018Breaking the gridlock in Mixture-of-Experts: Consistent and Efficient Algorithms
  6. 2025Mixture of Experts Provably Detect and Learn the Latent Cluster Structure in Gradient-Based Learning
  7. 2024Beyond Parameter Count: Implicit Bias in Soft Mixture of Experts
  8. 2023Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts