mlsanityData to 14 Oct 2026

AcceptMoE: Commitment-Weighted Self-Sizing Verifier Expert Sets for Efficient MoE Speculative Decoding

arXiv 2608.02989 ↗First posted 4 Aug 2026cs.LG cs.CL cs.DCcited by 1

On the map: Mixture-of-experts → Efficient MoE training and inference systems

also in 1 more
Abstract (arXiv)

Speculative decoding verifies a tree of draft tokens in one target-model forward pass. For a mixture-of-experts (MoE) target, however, parallel verification can activate the union of the experts selected by all tree nodes, even though only a small subset of those nodes reaches the accepted output. Token count, activated-expert union size, and expert-weight traffic are therefore distinct cost measures: reducing the token workload need not shrink the expert union proportionally, and under offloading, transfer traffic also depends on cache residency. We introduce AcceptMoE, a verifier-side expert selector that combines target-router scores with offline-estimated commitment probabilities and automatically adjusts the number of eligible experts for each verification block, eliminating the need for a user-specified expert budget. Under offloading, AcceptMoE conditions expert eligibility on cache residency instead of predicting natural routes and prefetching the corresponding expert weights. Although constraining target-expert eligibility changes the model distribution, across 12 model-task pairs spanning three MoE targets and four benchmarks, AcceptMoE's mean accuracy is 0.27 percentage points lower than that of EAGLE-3 speculative decoding with natural routing. Served with SGLang at batch size one, it reaches 1.290 times the throughput of this baseline with all expert weights in GPU memory, and 2.06 times under physical expert offloading, while reducing host-to-device traffic by 73.6 percent to 77.1 percent.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Efficient MoE training and inference systems, one of 12 directions in Mixture-of-experts. Builds on 26 of its 30 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

26 of 30 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts14 Jul
  2. 2026Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decoding1 May
  3. 2026XShare: Collaborative in-Batch Expert Sharing for Faster MoE Inference6 Feb
  4. 2026Dynamic Expert Sharing: Decoupling Memory from Parallelism in Mixture-of-Experts Diffusion LLMs31 Jan
  5. 2025MoE-SpeQ: Speculative Quantized Decoding with Proactive Expert Prefetching and Offloading for Mixture-of-Experts18 Nov
  6. 2025SP-MoE: Speculative Decoding and Prefetching for Accelerating MoE-based Model Inference11 Oct
  7. 2025Utility-Driven Speculative Decoding for Mixture-of-Experts17 Jun
  8. 2025MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE26 May

Other references · 18

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  8. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  9. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  10. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  11. 2024EAGLE-2: Faster Inference of Language Models with Dynamic Draft Treescited by 370
  12. 2025EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Testcited by 331
6 more
  1. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  2. 2022Task-Specific Expert Pruning for Sparse Mixture-of-Expertscited by 72
  3. 2024Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Expertscited by 37
  4. 2024Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inferencecited by 16
  5. 2024DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Modelscited by 15
  6. 2025MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skippingcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Efficient Mixture-of-Experts with Speculative Decoding via Expert Coactivationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MoE-Spec: Expert Budgeting for Efficient Speculative Decoding
  2. 2026SpecFit: Not All Draft Tokens Cost the Same in Offloaded MoE Inference
  3. 2026DraftExpert: Expansion-Aware Self-Speculative Decoding for End-Device MoE Inference
  4. 2026The Limits of Speculation: Bounding Speculative Decoding in Mixture-of-Experts
  5. 2026VIA-SD: Verification via Intra-Model Routing for Speculative Decoding
  6. 2026MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
  7. 2026BigMoMo: Efficient Inference of Large-Scale MoE with Speculative Decoding on Mobile Devices
  8. 2026S2-MoE: Enabling Efficient Self-Speculative Decoding for Mixture-of-Experts on Edge Devices