Detecting a Route Flip Is Easier Than Knowing Whether to Fix It: Causal Route-Mediated Damage in Quantized Mixture-of-Experts
On the map: Mixture-of-experts → Expert routing strategies
also in 1 more
Abstract (arXiv)
Top-k Mixture-of-Experts (MoE) routing is discontinuous, so a deployment-motivated numerical disturbance -- simulated 4-bit KV-cache quantization read by a protected BF16 gate -- pushes tokens across decision boundaries and flips which experts fire. This paper proposes no new mitigation; it supplies a causal apparatus, empirical findings, and a detection-limit result. A four-run apparatus prices the route-mediated fraction (RMF) of quantization damage, a token-level attribution decomposes it by mechanism, and pre-registered probes carry the findings across three architectures. On OLMoE-1B-7B at 4-bit KV (pilot), about a third of the damage is routing-mediated: RMF ~ 0.31 (discovery 0.31 [0.20, 0.41]; process-replicated mean 0.313 +/- 0.020; pre-registered re-execution 0.231). The deployable router margin detects that a flip occurred (AUC 0.772) but cannot tell a harmful flip from a helpful one (at chance): among the tested local, inference-observable router statistics we find no predictor of a flip's loss sign above chance -- an empirical benefit-detection barrier bounding selective repair restricted to this feature family. The signed-flip tax and sign-inseparability carry cross-model; the clean-reference remedy's payout is architecture-modulated; a controlled same-checkpoint flag-swap re-scopes the gate's normalization convention to a damage-magnitude moderator, not a route-recoverability mechanism. A real int4 KV kernel yields a fraction compatible with the fake-quant dose curve but underpowered (95% CI [-0.111, 0.394] includes zero) -- ruling out gross disagreement, not an independent replication. Hypotheses, thresholds, and evaluations were pre-registered before measurement, with misses reported; a pre-registered held-out read replicates the partition and the near-cancelling tax out of sample, while the strict impossibility exclusion narrowly misses.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts. Builds on 38 of its 39 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
38 of 39 references.
Closest ancestors · 8
Same direction, newest first
- 2026Geometric and Stochastic Analysis of Discontinuities in Sparse Mixture-of-Experts17 Jun
- 2026SoftMoE: Soft Differentiable Routing for Mixture-of-Experts in LLMs16 Jun
- 2026From Observation to Intervention: A Causal Audit of Expert Importance in Mixture-of-Experts Models9 Jun
- 2026Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts12 May
- 2026Routing-Free Mixture-of-Experts1 Apr
- 2026Is Retraining-Free Enough? The Necessity of Router Calibration for Efficient MoE Compression10 Feb
- 2022Mixture-of-Experts with Expert Choice Routing18 Feb
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity11 Jan
Other references · 30
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2025Qwen3 Technical Reportcited by 8,195
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2024Qwen2 Technical Reportcited by 2,076
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
- 2024Mixtral of Expertscited by 1,785
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
18 more
- 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 834
- 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
- 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
- 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
- 2021Causal Abstractions of Neural Networkscited by 401
- 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 391
- 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
- 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
- 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
- 2023Localizing Model Behavior with Path Patchingcited by 182
- 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 165
- 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 155
- 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inferencecited by 36
- 2025EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimizationcited by 9
- 2025MixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoningcited by 8
- 2026GEMQ: Global Expert-Level Mixed-Precision Quantization for MoE LLMscited by 5
- 2026Value-and-Structure Alignment for Routing-Consistent Quantization of Mixture-of-Experts Modelscited by 5
- 2025KVmix: Gradient-Based Layer Importance-Aware Mixed-Precision Quantization for KV Cachecited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026A Persistent State for Auditable Mixture-of-Experts Routing
- 2026Safety-Oriented Routing Analysis of Mixtral MoE Under Benign and Harmful Prompts
- 2026Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs
- 2026When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
- 2026Route-Block Membership Selects Packed-AWQ Arithmetic: A Controlled Single-Fixture Mechanism Study
- 2026Adaptive Inverted-Index Routing for Granular Mixtures-of-Experts
- 2026Router Sensitivity Under Lightweight Fine-Tuning Identifies Prunable Experts in Mixture-of-Experts Models
- 2026Router Prior Bias: Preserving Base Routing Structure in MoE Post-Training