mlsanityData to 14 Oct 2026

Is Retraining-Free Enough? The Necessity of Router Calibration for Efficient MoE Compression

arXiv 2603.02217 ↗First posted 10 Feb 2026cs.LG cs.AIcited by 4

On the map: Mixture-of-experts → Expert routing strategies

Abstract (arXiv)

Mixture-of-Experts (MoE) models scale capacity efficiently, but their massive parameter footprint creates a deployment-time memory bottleneck. We organize retraining-free MoE compression into three paradigms - Expert Pruning, Expert Editing, and Expert Merging - and show that persistent post-compression degradation largely stems from a neglected factor: router-expert mismatch when experts are changed but the router is left untouched. We argue that effective retraining-free compression should avoid updating expert parameters while allowing lightweight router calibration. To this end, we propose Router Knowledge Distillation (Router KD), which updates only a tiny fraction of parameters (the router) by distilling the original model's next-token distribution on unlabeled calibration data. Experiments across representative methods in all three paradigms demonstrate consistent performance recovery, with substantially larger gains in fine-grained MoEs (many small experts) than in coarse-grained MoEs due to their more complex routing decision boundaries.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts. Builds on 41 of its 54 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

41 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations9 Apr
  2. 2023Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy2 Oct
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity11 Jan

Other references · 38

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  10. 2021Program Synthesis with Large Language Modelscited by 3,637
  11. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  12. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
26 more
  1. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  2. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  5. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  7. 2018CoQA: A Conversational Question Answering Challengecited by 683
  8. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  9. 2019ELI5: Long Form Question Answeringcited by 575
  10. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  11. 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelscited by 146
  12. 2024MoE-I2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decompositioncited by 57
  13. 2024Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costscited by 53
  14. 2025REAP the Experts: Why Pruning Prevails for One-Shot MoE compressioncited by 44
  15. 2024Retraining-Free Merging of Sparse MoE via Hierarchical Clusteringcited by 35
  16. 2025Delta Decompression for MoE-based LLMs Compressioncited by 33
  17. 2025DiEP: Adaptive Mixture-of-Experts Compression through Differentiable Expert Pruningcited by 26
  18. 2025Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMscited by 25
  19. 2024Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniquescited by 24
  20. 2025Finding Fantastic Experts in MoEs: A Unified Study for Expert Dropping Strategies and Observationscited by 20
  21. 2025EAC-MoE: Expert-Selection Aware Compressor for Mixture-of-Experts Large Language Modelscited by 18
  22. 2025MoNE: Replacing Redundant Experts with Lightweight Novices for Structured Pruning of MoEcited by 13
  23. 2025MoLAE: Mixture of Latent Experts for Parameter-Efficient Language Modelscited by 10
  24. 2025MoBE: Mixture-of-Basis-Experts for Compressing MoE-based LLMscited by 9
  25. 2025PuzzleMoE: Efficient Compression of Large Mixture-of-Experts Models via Sparse Expert Merging and Bit-packed inferencecited by 9
  26. 2025Expert Merging: Model Merging with Unsupervised Expert Alignment and Importance-Guided Layer Chunkingcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Detecting a Route Flip Is Easier Than Knowing Whether to Fix It: Causal Route-Mediated Damage in Quantized Mixture-of-Expertscited by 2
  2. 2026Router Sensitivity Under Lightweight Fine-Tuning Identifies Prunable Experts in Mixture-of-Experts Modelscited by 0
  3. 2026Beyond Retraining-Free MoE Compression: A Cost-Normalized Study of Post-Compression Adjustmentcited by 0
  4. 2026RAZOR: Pruning Replaceable Experts in LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ConMoE: Expert-Pool Consolidation via Prototype Reassignment for MoE Compression
  2. 2026Router Prior Bias: Preserving Base Routing Structure in MoE Post-Training
  3. 2026ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference
  4. 2026Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation
  5. 2024ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing
  6. 2026MoRA: MoE Pruning via Router Bias Learning and Expert Approximation
  7. 2026When Model Merging Breaks Routing: Training-Free Calibration for MoE
  8. 2026Beyond Routing: Decoupling Expert Dispatch and Aggregation in Sparse Mixture-of-Experts