mlsanityData to 7 Oct 2026

MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Models

arXiv 2405.18832 ↗First posted 29 May 2024cs.LG cs.AI cs.ARcited by 11

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Mixture-of-Experts (MoE) large language models (LLM) have memory requirements that often exceed the GPU memory capacity, requiring costly parameter movement from secondary memories to the GPU for expert computation. In this work, we present Mixture of Near-Data Experts (MoNDE), a near-data computing solution that efficiently enables MoE LLM inference. MoNDE reduces the volume of MoE parameter movement by transferring only the textithot experts to the GPU, while computing the remaining textitcold experts inside the host memory device. By replacing the transfers of massive expert parameters with the ones of small activations, MoNDE enables far more communication-efficient MoE inference, thereby resulting in substantial speedups over the existing parameter offloading frameworks for both encoder and decoder operations.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 7 of its 15 references that are in the map. First posted on arXiv 29 May 2024.

Builds on

7 of 15 references.

References · 7

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2021ZeRO-Offload: Democratizing Billion-Scale Model Trainingcited by 339
  6. 2021ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learningcited by 253
  7. 2023Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inferencecited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 6.

11 papers.

  1. 2024A Survey on Inference Optimization Techniques for Mixture of Experts Modelscited by 46
  2. 2024Large Language Model Inference Acceleration: A Comprehensive Hardware Perspectivecited by 36
  3. 2025UbiMoE: A Ubiquitous Mixture-of-Experts Vision Transformer Accelerator With Hybrid Computation Pattern on FPGAcited by 4
  4. 2024FLARE: FP-Less PTQ and Low-ENOB ADC Based AMS-PiM for Error-Resilient, Fast, and Efficient Transformer Accelerationcited by 0
  5. 2025CoQMoE: Co-Designed Quantization and Computation Orchestration for Mixture-of-Experts Vision Transformer on FPGAcited by 0
  6. 2026A Scheduling Framework for Efficient MoE Inference on Edge GPU-NDP Systemscited by 0
  7. 2026Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Schedulingcited by 0
  8. 2026HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inferencecited by 0
  9. 2026MoE Expert Execution in Disaggregated LLM Serving with a High-Bandwidth ReRAM Near-Memory Architecturecited by 0
  10. 2026A Multi-Engine Dataflow for MoE Decoding on Scratchpad-Based Tensor Acceleratorscited by 0
  11. 2026MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Models
  2. 2023Fast Inference of Mixture-of-Experts Language Models with Offloading
  3. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
  4. 2025PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inference
  5. 2025Accelerating Mixture-of-Expert Inference with Adaptive Expert Split Mechanism
  6. 2024Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques
  7. 2025OD-MoE: On-Demand Expert Loading for Cacheless Edge-Distributed MoE Inference
  8. 2025Context-Aware Mixture-of-Experts Inference on CXL-Enabled GPU-NDP Systems