mlsanityData to 14 Oct 2026

MoE Expert Execution in Disaggregated LLM Serving with a High-Bandwidth ReRAM Near-Memory Architecture

arXiv 2608.13962 ↗First posted 14 Aug 2026cs.ARcited by 0

On the map: Mixture-of-experts → Efficient MoE training and inference systems

also in 1 more
Abstract (arXiv)

Attention-FFN disaggregation maps LLM modules to specialized pools, creating an opening to keep Mixture-of-Experts (MoE) weights resident in a high-bandwidth FFN pool. Decode SLOs, however, cap the run-batch while sparse routing expands the activated-expert union, so weight traffic amortizes poorly and routing skew idles cold-expert resources. The FFN pool must therefore deliver weight-read bandwidth density under sparse unions and recover occupancy under skew without a global sharing fabric. We present a ReRAM near-memory architecture that keeps expert weights resident behind high-bandwidth local reads. The design factors actual MFU into ideal MFU and occupancy, recovers occupancy with bounded core-local multicast pooling, coactivation-aware placement, and load-aware fetch, and sizes each communication level from induced demand. A measured + modeled study on Qwen3.5-35B-A3B, Qwen3.5-397B-A17B, and GLM-5.2 shows that side-4 pooling raises occupancy from 0.328 to 0.519 and, at iso-peak compute, lowers per-token FFN-pool latency by 9.5x versus H20 with 20x lower weight-movement energy; an H20-attention + ReRAM-FFN system reduces decode TPOT by 1.25-4.0x, 2.4-10.3x, and 2.5-10.4x versus a homogeneous H20 pool.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 21 of its 37 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

21 of 37 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference7 Oct
  2. 2024MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs18 Nov
  3. 2024Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batching2 Sep
  4. 2024MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Models29 May
  5. 2023SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models29 Oct
  6. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference23 Aug
  7. 2022Accelerating Distributed MoE Training and Inference with Lina31 Oct
  8. 2022Tutel: Adaptive Mixture-of-Experts at Scale7 Jun

Other references · 13

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2022Mixture-of-Experts with Expert Choice Routingcited by 623
  3. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  4. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  5. 2022Efficiently Scaling Transformer Inferencecited by 539
  6. 2019PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inferencecited by 61
  7. 2024NeuPIMs: NPU-PIM Heterogeneous Acceleration for Batched LLM Inferencingcited by 52
  8. 2025PIM Is All You Need: A CXL-Enabled GPU-Free System for Large Language Model Inferencecited by 22
  9. 2025PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing Systemcited by 20
  10. 2024IANUS: Integrated Accelerator based on NPU-PIM Unified Memory Systemcited by 13
  11. 2020TIMELY: Pushing Data Movements and Interfaces in PIM Accelerators Towards Local and in Time Domaincited by 12
  12. 2025CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIMcited by 5
1 more
  1. 2026Ouroboros: Wafer-Scale SRAM CIM with Token-Grained Pipelining for Large Language Model Inferencecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ExpertPlex: A High-Goodput Disaggregated Serving System for MoE LLMs with Adaptive Persistent Kernels
  2. 2026Revealing the Challenges of Attention-FFN Disaggregation for Modern MoE Models and Hardware Systems
  3. 2026Beyond Capacity: Scalable MoE LLM Inference via High-Bandwidth Flash with Direct GPU and HBM Paths
  4. 2026How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
  5. 2026ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference
  6. 2025Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining
  7. 2026ELDR: Expert-Locality-Aware Decode Routing for PD-Disaggregated MoE Serving
  8. 2024Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design