mlsanityData to 7 Oct 2026

Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation

arXiv 2512.17073 ↗First posted 18 Dec 2025cs.LGcited by 3

On the map: Mixture-of-experts → Efficient MoE training and inference systems

Abstract (arXiv)

Mixture-of-Experts (MoE) models scale capacity via sparse activation but stress memory and bandwidth. Offloading alleviates GPU memory by fetching experts on demand, yet token-level routing causes irregular transfers that make inference I/O-bound. Static uniform quantization reduces traffic but degrades accuracy under aggressive compression by ignoring expert heterogeneity. We present Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation, which performs router-guided precision restoration using precomputed low-rank compensators. At inference time, our method transfers compact low-rank factors with Top-n (n<k) experts per token and applies compensation to them, keeping others low-bit. Integrated with offloading on GPU and GPU-NDP systems, our method delivers a superior bandwidth-accuracy trade-off and improved throughput.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 22 of its 37 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

22 of 37 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs18 Nov
  2. 2024HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference3 Nov
  3. 2024ProMoE: Fast MoE-based LLM Serving using Proactive Caching29 Oct
  4. 2024Read-ME: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design24 Oct
  5. 2024Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batching2 Sep
  6. 2024Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models10 Feb
  7. 2023Fast Inference of Mixture-of-Experts Language Models with Offloading28 Dec
  8. 2023Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference23 Aug

Other references · 14

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  7. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  8. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
  9. 2020Robust Quantization: One Model to Rule Them Allcited by 63
  10. 2024Compressing Large Language Models using Low Rank and Low Precision Decompositioncited by 58
  11. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 39
  12. 2025MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Designcited by 26
2 more
  1. 2025KurTail : Kurtosis-based LLM Quantizationcited by 12
  2. 2025MoPEQ: Mixture of Mixed Precision Quantized Expertscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Harvest: Opportunistic Peer-to-Peer GPU Caching for LLM Inferencecited by 2
  2. 2026Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligencecited by 0
  3. 2026Residual Sparsification via Output Importance for Compressing Mixture-of-Experts LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Dynamic Expert Quantization for Scalable Mixture-of-Experts Inference
  2. 2025MoBiLE: Efficient Mixture-of-Experts Inference on Consumer GPU with Mixture of Big Little Experts
  3. 2026Mira: Memory-Efficient MoE Inference Using Adaptive Caching and Predictive Expert Staging
  4. 2026Every Expert Counts: ExactMoE for Memory-Efficient W4A16 Inference
  5. 2024ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
  6. 2026MELINOE: Fine-Tuning Enables Memory-Efficient Inference for Mixture-of-Experts Models
  7. 2026WiSP: A Working-Set View of Mixture-of-Experts Serving on Extremely Low-Resource Hardware
  8. 2024Communication-Efficient Sparsely-Activated Model Training via Sequence Migration and Token Condensation