# SliceMoE: Bit-Sliced Expert Caching under Miss-Rate Constraints for Efficient MoE Inference

arXiv: https://arxiv.org/abs/2512.12990 · HTML: https://mlsanity.com/paper/2512.12990/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts.
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 15 Dec 2025.

## Placed in

- Mixture-of-experts / [Efficient MoE training and inference systems](https://mlsanity.com/moe/efficient-moe-training-and-inference-systems/)

## Builds on (most cited first, 26 of 26)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference](https://mlsanity.com/paper/2308.12066/) (2023)
- [HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference](https://mlsanity.com/paper/2411.01433/) (2024)
- [Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs](https://mlsanity.com/paper/2402.10517/) (2024)
- [AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE Inference](https://mlsanity.com/paper/2408.10284/) (2024)
- [ProMoE: Fast MoE-based LLM Serving using Proactive Caching](https://mlsanity.com/paper/2410.22134/) (2024)
- [Merging Experts into One: Improving Computational Efficiency of Mixture of Experts](https://mlsanity.com/paper/2310.09832/) (2023)
- [Delta Decompression for MoE-based LLMs Compression](https://mlsanity.com/paper/2502.17298/) (2025)
- [D2MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving](https://mlsanity.com/paper/2504.15299/) (2025)
- [OMoE: Diversifying Mixture of Low-Rank Adaptation by Orthogonal Finetuning](https://mlsanity.com/paper/2501.10062/) (2025)
- [eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference](https://mlsanity.com/paper/2503.06823/) (2025)
- [MaskMoE: Boosting Token-Level Learning via Routing Mask in Mixture-of-Experts](https://mlsanity.com/paper/2407.09816/) (2024)
- [Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling](https://mlsanity.com/paper/2503.04398/) (2025)
- [DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts](https://mlsanity.com/paper/2506.09351/) (2025)
- [From Score Distributions to Balance: Plug-and-Play Mixture-of-Experts Routing](https://mlsanity.com/paper/2510.03293/) (2025)
- [BuddyMoE: Exploiting Expert Redundancy to Accelerate Memory-Constrained Mixture-of-Experts Inference](https://mlsanity.com/paper/2511.10054/) (2025)
- [SSD Offloading for LLM Mixture-of-Experts Weights Considered Harmful in Energy Efficiency](https://mlsanity.com/paper/2508.06978/) (2025)
- [S'MoRE: Structural Mixture of Residual Experts for Parameter-Efficient LLM Fine-tuning](https://mlsanity.com/paper/2504.06426/) (2025)
- [SMoE: An Algorithm-System Co-Design for Pushing MoE to the Edge via Expert Substitution](https://mlsanity.com/paper/2508.18983/) (2025)

## Built on by (most cited first, 3 of 3)

- [StagQ: Constraint-Driven Multi-Precision Weight Quantization for LLMs](https://mlsanity.com/paper/2610.05977/) (2026)
- [ELMoE-3D: Leveraging Intrinsic Elasticity of MoE for Hybrid-Bonding-Enabled Self-Speculative Decoding in On-Premises Serving](https://mlsanity.com/paper/2604.14626/) (2026)
- [DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving](https://mlsanity.com/paper/2610.13328/) (2026)

## Abstract (arXiv)

MoE models offer efficient scaling through conditional computation, but their large parameter size and expensive expert offloading make on-device deployment challenging. Existing acceleration techniques such as prefetching or expert clustering often increase energy usage or reduce expert diversity. We present SliceMoE, an energy-efficient MoE inference framework for miss-rate-constrained deployment. SliceMoE introduces Dynamic Bit-Sliced Caching (DBSC), which caches experts at slice-level granularity and assigns precision on demand to expand effective expert capacity. To support mixed-precision experts without memory duplication, we propose Calibration-Free Asymmetric Matryoshka Quantization (AMAT), a truncation-based scheme that maintains compatibility between low-bit and high-bit slices. We further introduce Predictive Cache Warmup (PCW) to reduce early-decode cold misses by reshaping cache contents during prefill. Evaluated on DeepSeek-V2-Lite and Qwen1.5-MoE-A2.7B, SliceMoE reduces decode-stage energy consumption by up to 2.37x and 2.85x, respectively, and improves decode latency by up to 1.81x and 1.64x, while preserving near-high-bit accuracy. These results demonstrate that slice-level caching enables an efficient on-device MoE deployment.