# MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Models

arXiv: https://arxiv.org/abs/2405.18832 · HTML: https://mlsanity.com/paper/2405.18832/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts.
- Builds on 7 of its 15 references that are in the map.
- First posted on arXiv 29 May 2024.

## Placed in

- Mixture-of-experts / [Efficient MoE training and inference systems](https://mlsanity.com/moe/efficient-moe-training-and-inference-systems/)

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [ZeRO-Offload: Democratizing Billion-Scale Model Training](https://mlsanity.com/paper/2101.06840/) (2021)
- [ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning](https://mlsanity.com/paper/2104.07857/) (2021)
- [Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference](https://mlsanity.com/paper/2303.06182/) (2023)

## Built on by (most cited first, 11 of 11)

- [A Survey on Inference Optimization Techniques for Mixture of Experts Models](https://mlsanity.com/paper/2412.14219/) (2024)
- [Large Language Model Inference Acceleration: A Comprehensive Hardware Perspective](https://mlsanity.com/paper/2410.04466/) (2024)
- [UbiMoE: A Ubiquitous Mixture-of-Experts Vision Transformer Accelerator With Hybrid Computation Pattern on FPGA](https://mlsanity.com/paper/2502.05602/) (2025)
- [FLARE: FP-Less PTQ and Low-ENOB ADC Based AMS-PiM for Error-Resilient, Fast, and Efficient Transformer Acceleration](https://mlsanity.com/paper/2411.14733/) (2024)
- [CoQMoE: Co-Designed Quantization and Computation Orchestration for Mixture-of-Experts Vision Transformer on FPGA](https://mlsanity.com/paper/2506.08496/) (2025)
- [A Scheduling Framework for Efficient MoE Inference on Edge GPU-NDP Systems](https://mlsanity.com/paper/2601.03992/) (2026)
- [Expert Streaming: Accelerating Low-Batch MoE Inference via Multi-chiplet Architecture and Dynamic Expert Trajectory Scheduling](https://mlsanity.com/paper/2603.27624/) (2026)
- [HCRMap: Pressure-Aware Hot-Expert Residency Mapping for 3.5D MoE Chiplet Inference](https://mlsanity.com/paper/2607.11586/) (2026)
- [MoE Expert Execution in Disaggregated LLM Serving with a High-Bandwidth ReRAM Near-Memory Architecture](https://mlsanity.com/paper/2608.13962/) (2026)
- [A Multi-Engine Dataflow for MoE Decoding on Scratchpad-Based Tensor Accelerators](https://mlsanity.com/paper/2609.21137/) (2026)
- [MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inference](https://mlsanity.com/paper/2609.34077/) (2026)

## Abstract (arXiv)

Mixture-of-Experts (MoE) large language models (LLM) have memory requirements that often exceed the GPU memory capacity, requiring costly parameter movement from secondary memories to the GPU for expert computation. In this work, we present Mixture of Near-Data Experts (MoNDE), a near-data computing solution that efficiently enables MoE LLM inference. MoNDE reduces the volume of MoE parameter movement by transferring only the textithot experts to the GPU, while computing the remaining textitcold experts inside the host memory device. By replacing the transfers of massive expert parameters with the ones of small activations, MoNDE enables far more communication-efficient MoE inference, thereby resulting in substantial speedups over the existing parameter offloading frameworks for both encoder and decoder operations.