# Half the Experts, All the Code: One-Shot Domain Pruning of Mixture-of-Experts LLMs for Coding

arXiv: https://arxiv.org/abs/2607.16721 · HTML: https://mlsanity.com/paper/2607.16721/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in MoE large language models, one of 12 directions in Mixture-of-experts.
- Builds on 10 of its 13 references that are in the map.
- First posted on arXiv 18 Jul 2026.

## Placed in

- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)

## Builds on (most cited first, 10 of 10)

- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [REAP the Experts: Why Pruning Prevails for One-Shot MoE compression](https://mlsanity.com/paper/2510.13999/) (2025)
- [Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations](https://mlsanity.com/paper/2504.06792/) (2025)
- [Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models](https://mlsanity.com/paper/2504.07807/) (2025)
- [MoE Pathfinder: Trajectory-driven Expert Pruning](https://mlsanity.com/paper/2512.18425/) (2025)
- [How to Score Experts for One-Shot MoE Expert Pruning: A Unified Formulation and Selection Principle](https://mlsanity.com/paper/2606.15716/) (2026)
- [Extracting Small Translation Specialists from LLMs by Aggressively Pruning Experts](https://mlsanity.com/paper/2605.28042/) (2026)
- [Less is MoE: Trimming Experts in Domain-Specialist Language Models](https://mlsanity.com/paper/2606.05538/) (2026)
- [It Takes a MAESTRO To Prune Bad Experts](https://mlsanity.com/paper/2607.08601/) (2026)
- [On the Utility and Factual Reliability of Pruned Mixture-of-Experts Models in the Biomedical Domain](https://mlsanity.com/paper/2607.01444/) (2026)

## Built on by (most cited first, 3 of 3)

- [When Compression Scores Cannot Decide: Information Boundaries for Group-Robust LLM Pruning](https://mlsanity.com/paper/2608.02940/) (2026)
- [Decorrelation Is Not Complementarity: Skill, Not Lineage, Governs Trusted-Monitor Ensembles](https://mlsanity.com/paper/2608.16190/) (2026)
- [RAZOR: Pruning Replaceable Experts in LLMs](https://mlsanity.com/paper/2609.30465/) (2026)

## Abstract (arXiv)

The strongest open-weight coding models are mixture-of-experts (MoE) networks: most of their size comes from large pools of "expert" subnetworks, of which only a few act on any token. That pool is why these models do not fit on the machines most developers own, yet for a user who only wants coding help, most experts encode abilities that will never be invoked. We ask how many experts can be removed, and which, by pruning two recent open-weight MoE models from different families (Qwen3.6-35B-A3B and Gemma-4-26B-A4B) under five selection strategies, judged the way a user would: by whether the model still writes correct code. Half the experts can be removed from either model with no statistically detectable loss on the primary code benchmark, and the damage lands almost entirely on abilities outside coding, the intended trade. But the winning strategy flips between the two models, so a recipe validated on one family cannot be assumed to work on another. We further show that perplexity, the metric much of the pruning literature leans on, can rate a broken model above an intact one; that a lightweight fine-tune recovers about half of what aggressive pruning loses; and that against quantizing the full model to the same memory, pruning wins only where quantization would have to drop below 3 bits per weight. Five attempts to overturn that crossover, with failure criteria fixed in advance (better calibration, guarded selection, causal expert importance, failure attribution, and an agentic evaluation letting each model repair its failures from execution feedback), all leave it standing; the last shows single-shot benchmarks overstate compression penalties broadly, as one repair turn erases the 2-bit quantization penalty entirely. Expert pruning works, but it demands per-model validation on the task the model will actually serve.