# AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE Inference

arXiv: https://arxiv.org/abs/2408.10284 · HTML: https://mlsanity.com/paper/2408.10284/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts.
- Builds on 11 of its 21 references that are in the map.
- First posted on arXiv 19 Aug 2024.

## Placed in

- Mixture-of-experts / [Efficient MoE training and inference systems](https://mlsanity.com/moe/efficient-moe-training-and-inference-systems/)

## Builds on (most cited first, 11 of 11)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale](https://mlsanity.com/paper/2207.00032/) (2022)
- [ZeRO-Offload: Democratizing Billion-Scale Model Training](https://mlsanity.com/paper/2101.06840/) (2021)
- [PyHessian: Neural Networks Through the Lens of the Hessian](https://mlsanity.com/paper/1912.07145/) (2019)
- [ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning](https://mlsanity.com/paper/2104.07857/) (2021)
- [Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference](https://mlsanity.com/paper/2308.12066/) (2023)
- [Fast Inference of Mixture-of-Experts Language Models with Offloading](https://mlsanity.com/paper/2312.17238/) (2023)

## Built on by (most cited first, 30 of 42)

- [HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference](https://mlsanity.com/paper/2411.01433/) (2024)
- [A Survey on Inference Optimization Techniques for Mixture of Experts Models](https://mlsanity.com/paper/2412.14219/) (2024)
- [Delta Decompression for MoE-based LLMs Compression](https://mlsanity.com/paper/2502.17298/) (2025)
- [D2MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving](https://mlsanity.com/paper/2504.15299/) (2025)
- [MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE](https://mlsanity.com/paper/2505.19645/) (2025)
- [HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference](https://mlsanity.com/paper/2504.05897/) (2025)
- [Mixture of Cache-Conditional Experts for Efficient Mobile Device Inference](https://mlsanity.com/paper/2412.00099/) (2024)
- [DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference](https://mlsanity.com/paper/2501.10375/) (2024)
- [SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference](https://mlsanity.com/paper/2510.10302/) (2025)
- [Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling](https://mlsanity.com/paper/2503.04398/) (2025)
- [Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism](https://mlsanity.com/paper/2509.08342/) (2025)
- [Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models](https://mlsanity.com/paper/2505.16056/) (2025)
- [LayerScope: Predictive Cross-Layer Scheduling for Efficient Multi-Batch MoE Inference on Legacy Servers](https://mlsanity.com/paper/2509.23638/) (2025)
- [SlimCaching: Edge Caching of Mixture-of-Experts for Distributed Inference](https://mlsanity.com/paper/2507.06567/) (2025)
- [Accelerating Edge Inference for Distributed MoE Models with Latency-Optimized Expert Placement](https://mlsanity.com/paper/2508.12851/) (2025)
- [Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference](https://mlsanity.com/paper/2511.15015/) (2025)
- [Effective MoE-based LLM Compression by Exploiting Heterogeneous Inter-Group Experts Routing Frequency and Information Density](https://mlsanity.com/paper/2602.09316/) (2026)
- [EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Models](https://mlsanity.com/paper/2505.23830/) (2025)
- [MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios](https://mlsanity.com/paper/2603.09983/) (2026)
- [DynaMo: Runtime Switchable Quantization for MoE with Cross-Dataset Adaptation](https://mlsanity.com/paper/2503.21135/) (2025)
- [Alloc-MoE: Budget-Aware Expert Activation Allocation for Efficient Mixture-of-Experts Inference](https://mlsanity.com/paper/2604.08133/) (2026)
- [SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inference](https://mlsanity.com/paper/2512.12990/) (2025)
- [SERE: Similarity-based Expert Re-routing for Efficient Batch Decoding in MoE Models](https://mlsanity.com/paper/2602.07616/) (2026)
- [OrderMoE: An expert similarity driven distributed edge MoE inference](https://mlsanity.com/paper/2607.17154/) (2026)
- [A3D-MoE: Acceleration of Large Language Models with Mixture of Experts via 3D Heterogeneous Integration](https://mlsanity.com/paper/2507.19142/) (2025)
- [CoMoE: Collaborative Optimization of Expert Aggregation and Offloading for MoE-based LLMs at Edge](https://mlsanity.com/paper/2508.09208/) (2025)
- [DALI: A Workload-Aware Offloading Framework for Efficient MoE Inference on Local PCs](https://mlsanity.com/paper/2602.03495/) (2026)
- [DyMoE: Dynamic Expert Orchestration with Mixed-Precision Quantization for Efficient MoE Inference on Edge](https://mlsanity.com/paper/2603.19172/) (2026)
- [MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts](https://mlsanity.com/paper/2510.12357/) (2025)
- [OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference](https://mlsanity.com/paper/2512.03927/) (2025)

## Abstract (arXiv)

Mixture-of-Experts (MoE) models are designed to enhance the efficiency of large language models (LLMs) without proportionally increasing the computational demands. However, their deployment on edge devices still faces significant challenges due to high on-demand loading overheads from managing sparsely activated experts. This paper introduces AdapMoE, an algorithm-system co-design framework for efficient MoE inference. AdapMoE features adaptive expert gating and management to reduce the on-demand loading overheads. We observe the heterogeneity of experts loading across layers and tokens, based on which we propose a sensitivity-based strategy to adjust the number of activated experts dynamically. Meanwhile, we also integrate advanced prefetching and cache management techniques to further reduce the loading latency. Through comprehensive evaluations on various platforms, we demonstrate AdapMoE consistently outperforms existing techniques, reducing the average number of activated experts by 25% and achieving a 1.35x speedup without accuracy degradation. Code is available at: https://github.com/PKU-SEC-Lab/AdapMoE.