mlsanityData to 14 Oct 2026

Theory of Mixture-of-Experts for Mobile Edge Computing

arXiv 2412.15690 ↗First posted 20 Dec 2024cs.LGcited by 5

On the map: Mixture-of-experts → Expert routing strategies

Abstract (arXiv)

In mobile edge computing (MEC) networks, mobile users generate diverse machine learning tasks dynamically over time. These tasks are typically offloaded to the nearest available edge server, by considering communication and computational efficiency. However, its operation does not ensure that each server specializes in a specific type of tasks and leads to severe overfitting or catastrophic forgetting of previous tasks. To improve the continual learning (CL) performance of online tasks, we are the first to introduce mixture-of-experts (MoE) theory in MEC networks and save MEC operation from the increasing generalization error over time. Our MoE theory treats each MEC server as an expert and dynamically adapts to changes in server availability by considering data transfer and computation time. Unlike existing MoE models designed for offline tasks, ours is tailored for handling continuous streams of tasks in the MEC environment. We introduce an adaptive gating network in MEC to adaptively identify and route newly arrived tasks of unknown data distributions to available experts, enabling each expert to specialize in a specific type of tasks upon convergence. We derived the minimum number of experts required to match each task with a specialized, available expert. Our MoE approach consistently reduces the overall generalization error over time, unlike the traditional MEC approach. Interestingly, when the number of experts is sufficient to ensure convergence, adding more experts delays the convergence time and worsens the generalization error. Finally, we perform extensive experiments on real datasets in deep neural networks (DNNs) to verify our theoretical results.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts. Builds on 23 of its 34 references that are in the map. First posted on arXiv 20 Dec 2024.

Builds on

23 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity11 Jan

Other references · 22

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  3. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
  4. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  5. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  6. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 404
  7. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 278
  8. 2018Characterizing Implicit Bias in Terms of Optimization Geometrycited by 266
  9. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 209
  10. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 103
  11. 2022How catastrophic can catastrophic forgetting be in linear regression?cited by 81
  12. 2023A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexitycited by 77
10 more
  1. 2023Theory on Forgetting and Generalization of Continual Learningcited by 72
  2. 2022Beyond Not-Forgetting: Continual Learning with Backward Knowledge Transfercited by 59
  3. 2024Theory on Mixture-of-Experts in Continual Learningcited by 54
  4. 2023A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Trainingcited by 47
  5. 2022CoSCL: Cooperation of Small Continual Learners is Stronger than a Big Onecited by 43
  6. 2022MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Servicescited by 37
  7. 2024LocMoE: A Low-Overhead MoE for Large Language Model Trainingcited by 19
  8. 2024Toward Scalable Generative AI via Mixture of Experts in Mobile Edge Networkscited by 8
  9. 2018Follow Me at the Edge: Mobility-Aware Dynamic Service Placement for Mobile Edge Computingcited by 7
  10. 2020Pricing-Driven Service Caching and Task Offloading in Mobile Edge Computingcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  2. 2025MoE2: Optimizing Collaborative Inference for Edge Large Language Modelscited by 6
  3. 2025Plasticity-Aware Mixture of Experts for Learning Under QoE Shifts in Adaptive Video Streamingcited by 4
  4. 2026Don't Collapse Your Features: Why CenterLoss Hurts OOD Detection and Multi-Scale Mahalanobis Winscited by 0
  5. 2026AirMoE: Statistic-Augmented Over-the-Air MoE for Collaborative Intelligencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Efficient Federated Learning of Networked Mixture-of-Experts for Mobile Edge Computing
  2. 2025Mixture-of-Experts for Distributed Edge Computing with Channel-Aware Gating Function
  3. 2025CoMoE: Collaborative Optimization of Expert Aggregation and Offloading for MoE-based LLMs at Edge
  4. 2025FLEX-MoE: Federated Mixture-of-Experts with Load-balanced Expert Assignment for Edge Computing
  5. 2025Stable-MoE: Lyapunov-based Token Routing for Distributed Mixture-of-Experts Training over Edge Networks
  6. 2026APEX: Adaptive Expert Prefetching for Memory-Efficient Edge MoE Inference
  7. 2023Learning to Specialize: Joint Gating-Expert Training for Adaptive MoEs in Decentralized Settings
  8. 2026Towards a Statistical Understanding of Mixture-of-Experts