mlsanityData to 7 Oct 2026

Efficient MoE training and inference systems

386 papers5 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: moe, inference, expert, memory, gpu

Key facts. Efficient MoE training and inference systems: 386 papers on arXiv since 2020 (data through 7 Oct 2026). 107 of its 386 mapped papers (28%) cite “DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale” (2022, arXiv 2201.05596). 86 of its 386 mapped papers (22%) cite “Tutel: Adaptive Mixture-of-Experts at Scale” (2022, arXiv 2206.03382). 25 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

25 new papers, 17 Sep – 7 Oct; 5 this week.

  1. This week · 5
  2. Democratizing MoE inference on commodity GPUs with CoMoE
  3. Expert Coupling in MoE Pretraining: Reducing All-to-All Overhead with Correlated Placement and Token Shuffling also fits: Expert routing strategies
  4. Stepped MoE: Segment-Level Routing with Configurable Inference Complexity also fits: Expert routing strategies
  5. Cascadia: Resident 975B MoE Inference on Eleven AI PCs
  6. CIPHER-MoE: Balancing Efficiency and Routing Fidelity in Trillion-Scale MoE Training also fits: Expert routing strategies
  7. Earlier · 20
  8. Universal Test-Time Training also fits: MoE large language models
  9. Efficient Expert-Parallel Communication on PCIe-Connected Consumer GPUs
  10. HAPMoE: Heterogeneity-Aware Automatic Parallelism Planning for Mixture-of-Experts Models Training
  11. Hardware-Native Joint Sparse-Quantization for Trillion-Scale Mixture-of-Experts
  12. Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging
All 25
  1. BASE: Batch-Aware Selection of Experts Using Predicted Removal Error for Efficient MoE Decoding
  2. Mixture-of-Kittens: MoE Megakernel for NVL72s
  3. TopoEP: Topology-Aware Load Balancing for Expert-Parallel MoE Training
  4. SlimWise: Decoupling Expert Pruning Across Prefill and Decode for Efficient MoE Serving
  5. MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inference also fits: Expert routing strategies
  6. CipherGenome: Homomorphic Inference for Genomic Mixture-of-Experts
  7. GenomeOcean Anywhere: Private WebGPU Inference for Genome MoEs
  8. OLED-MoE: Accelerating MoE-Based dLLM Inference via Inter-Iteration Locality-Aware Expert Offloading
  9. Paging the Experts: A Reproducible Characterization of Flash-Backed MoE Inference on iPhone
  10. Exact Quantile Balancing and Load-Error Injection for Mixture-of-Experts also fits: Expert routing strategies
  11. Logit-Aware MIMO AirComp for Distributed Mixture-of-Experts LLM Inference over Wireless Edge Networks also fits: Expert routing strategies
  12. Efficient Mixture-of-Experts with Speculative Decoding via Expert Coactivation
  13. Weave: Fine-Grained Dynamic SM Scheduling in an MoE Megakernel for Compute-Communication Overlap
  14. A Multi-Engine Dataflow for MoE Decoding on Scratchpad-Based Tensor Accelerators
  15. DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

Foundations

Most cited by papers here

  1. DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale2022cited by 107 here
  2. Tutel: Adaptive Mixture-of-Experts at Scale2022cited by 86 here
  3. Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference2023cited by 77 here
  4. Accelerating Distributed MoE Training and Inference with Lina2022cited by 67 here
  5. Fast Inference of Mixture-of-Experts Language Models with Offloading2023cited by 59 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 181 here
  2. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 155 here
  3. 2021FastMoE: A Fast Mixture-of-Expert Training Systemcited by 35 here
  4. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 107 here
  5. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 86 here
  6. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inferencecited by 77 here
  7. 2024AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE Inferencecited by 36 here
  8. 2024HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inferencecited by 36 here
16 more it builds on, not in this chain
  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 203 here
  2. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 66 here
  3. 2022Mixture-of-Experts with Expert Choice Routingcited by 47 here
  4. 2022Accelerating Distributed MoE Training and Inference with Linacited by 67 here
  5. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 53 here
  6. 2023FlexMoE: Scaling Large-scale Sparse Pre-trained Model Training via Dynamic Device Placementcited by 33 here
  7. 2023Fast Inference of Mixture-of-Experts Language Models with Offloadingcited by 59 here
  8. 2024Mixtral of Expertscited by 101 here
  9. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 89 here
  10. 2024MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cachecited by 32 here
  11. 2024Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Modelscited by 48 here
  12. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 77 here
  13. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 30 here
  14. 2024MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUscited by 45 here
  15. 2024DeepSeek-V3 Technical Reportcited by 70 here
  16. 2025Qwen3 Technical Reportcited by 68 here

The 24 most cited of 437.

Browse

30 of 386, ranked 6–35

  1. 6MegaBlocks: Efficient Sparse Training with Mixture-of-Experts202253
  2. 7Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models202448
  3. 8MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs202445
  4. 9AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE Inference202436
  5. 10HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference202436
  6. 11FastMoE: A Fast Mixture-of-Expert Training System202135
  7. 12MoE-Infinity: Efficient MoE Inference on Personal Machines with Sparsity-Aware Expert Cache202432
  8. 13ProMoE: Fast MoE-based LLM Serving using Proactive Caching202429
  9. 14MoESys: A Distributed and Efficient Mixture-of-Experts Training and Inference System for Internet Services202226
  10. 15A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training202324
All 30
  1. 16EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices202324
  2. 17Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts202522
  3. 18Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference202422
  4. 19SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models202321
  5. 20Klotski: Efficient Mixture-of-Expert Inference via Expert-Aware Multi-Batch Pipeline202520
  6. 21HetuMoE: An Efficient Trillion-scale Mixture-of-Expert Distributed Training System202220
  7. 22Taming Latency-Memory Trade-Off in MoE-Based LLM Serving via Fine-Grained Expert Offloading202518
  8. 23MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing202518
  9. 24Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batching202415
  10. 25Lancet: Accelerating Mixture-of-Experts Training via Whole Graph Computation-Communication Overlapping202414
  11. 26DAOP: Data-Aware Offloading and Predictive Pre-Calculation for Efficient MoE Inference202414
  12. 27MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism202513
  13. 28Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding202513
  14. 29A Survey on Inference Optimization Techniques for Mixture of Experts Models202413
  15. 30Lynx: Enabling Efficient MoE Inference through Dynamic Batch-Aware Expert Selection202412
  16. 31Fate: Fast Edge Inference of Mixture-of-Experts Models via Cross-Layer Gate202511
  17. 32HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference202511
  18. 33FSMoE: A Flexible and Scalable Training System for Sparse Mixture-of-Experts Models202510
  19. 34Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures202510
  20. 35MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production202510