mlsanityData to 7 Oct 2026

AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE Inference

arXiv 2408.10284 ↗First posted 19 Aug 2024cs.LGcited by 42

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Mixture-of-Experts (MoE) models are designed to enhance the efficiency of large language models (LLMs) without proportionally increasing the computational demands. However, their deployment on edge devices still faces significant challenges due to high on-demand loading overheads from managing sparsely activated experts. This paper introduces AdapMoE, an algorithm-system co-design framework for efficient MoE inference. AdapMoE features adaptive expert gating and management to reduce the on-demand loading overheads. We observe the heterogeneity of experts loading across layers and tokens, based on which we propose a sensitivity-based strategy to adjust the number of activated experts dynamically. Meanwhile, we also integrate advanced prefetching and cache management techniques to further reduce the loading latency. Through comprehensive evaluations on various platforms, we demonstrate AdapMoE consistently outperforms existing techniques, reducing the average number of activated experts by 25% and achieving a 1.35x speedup without accuracy degradation. Code is available at: https://github.com/PKU-SEC-Lab/AdapMoE.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 11 of its 21 references that are in the map. First posted on arXiv 19 Aug 2024.

Builds on

11 of 21 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Fast Inference of Mixture-of-Experts Language Models with Offloading28 Dec
  2. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference23 Aug

Other references · 9

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  6. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 403
  7. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  8. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 291
  9. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 253

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 20; 2026: 18.

42 papers.

Sort:

By area:

  1. 2024HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inferencecited by 53
  2. 2024A Survey on Inference Optimization Techniques for Mixture of Experts Modelscited by 46
  3. 2025Delta Decompression for MoE-based LLMs Compressioncited by 32
  4. 2025D2MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Servingcited by 17
  5. 2025MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoEcited by 17
  6. 2025HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inferencecited by 16
  7. 2024Mixture of Cache-Conditional Experts for Efficient Mobile Device Inferencecited by 14
  8. 2024DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inferencecited by 14
  9. 2025SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inferencecited by 12
  10. 2025Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Schedulingcited by 9
  11. 2025Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanismcited by 9
  12. 2025Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Modelscited by 8
30 more
  1. 2025LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Serverscited by 8
  2. 2025SlimCaching: Edge Caching of Mixture-of-Experts for Distributed Inferencecited by 7
  3. 2025Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placementcited by 7
  4. 2025Dynamic Expert Quantization for Scalable Mixture-of-Experts Inferencecited by 6
  5. 2026Effective MoE-based LLM Compression by Exploiting Heterogeneous Inter-Group Experts Routing Frequency and Information Densitycited by 6
  6. 2025EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Modelscited by 5
  7. 2026MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarioscited by 5
  8. 2025DynaMo: Runtime Switchable Quantization for MoE with Cross-Dataset Adaptationcited by 4
  9. 2026SERE: Similarity-based Expert Re-routing for Efficient Batch Decoding in MoE Modelscited by 3
  10. 2026Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inferencecited by 3
  11. 2026OrderMoE: An expert similarity driven distributed edge MoE inferencecited by 3
  12. 2025A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integrationcited by 2
  13. 2025CoMoE: Collaborative Optimization of Expert Aggregation and Offloading for MoE-based LLMs at Edgecited by 2
  14. 2026DALI: A Workload-Aware Offloading Framework for Efficient MoE Inference on Local PCscited by 2
  15. 2026DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edgecited by 2
  16. 2025MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Expertscited by 1
  17. 2025OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inferencecited by 1
  18. 2025SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inferencecited by 1
  19. 2025Efficient CPU-GPU Collaborative Inference for MoE-based LLMs on Memory-Limited Systemscited by 1
  20. 2026Beyond Uniform Experts: Cost-Aware Expert Execution for Efficient Multi-Device MoE Inferencecited by 1
  21. 2026Cacheable by Design? Training Mixture-of-Experts Routers for Locality Against the Edge Memory-Bandwidth Wall: A Pre-Registered Negative Result with a Systems Measurement Studycited by 1
  22. 2026Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Schedulingcited by 0
  23. 2026Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devicescited by 0
  24. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligencecited by 0
  25. 2026TrimMoE A communication aware and adaptive depth framework for distributed edge inferencecited by 0
  26. 2026APEX: Adaptive Expert Prefetching for Memory-Efficient Edge MoE Inferencecited by 0
  27. 2026PCoMoE: Shifting MoE Inference from Monolithic Expert Selection to Fine-Grained Path Compositioncited by 0
  28. 2026HDA-MoE: Hybrid Parallelism and Dynamic, Adaptive Scheduling for Mixture-of-Experts with 3D Near-Memory Processingcited by 0
  29. 2026Mixture-of-Experts Language Models Can Be Strong and Efficient Retrieverscited by 0
  30. 2026BASE: Batch-Aware Selection of Experts Using Predicted Removal Error for Efficient MoE Decodingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference
  2. 2026Fast MoE Inference via Predictive Prefetching and Expert Replication
  3. 2025Faster MoE LLM Inference for Extremely Large Models
  4. 2025Mixture of Experts in Large Language Models
  5. 2026A Spatio-Temporal Expert Prefetching Framework for Efficient MoE-based LLM Inference
  6. 2026Speculating Experts Accelerates Inference for Mixture-of-Experts
  7. 2024Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costs
  8. 2025Mixture-of-Experts Can Surpass Dense LLMs Under Strictly Equal Resource