mlsanityData to 7 Oct 2026

Expert routing strategies

323 papers7 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: routing, moe, experts, expert, mixture-of-experts

Key facts. Expert routing strategies: 323 papers on arXiv since 2019 (data through 7 Oct 2026). 192 of its 323 mapped papers (59%) cite “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity” (2021, arXiv 2101.03961). 88 of its 323 mapped papers (27%) cite “Mixture-of-Experts with Expert Choice Routing” (2022, arXiv 2202.09368). 27 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

27 new papers, 17 Sep – 7 Oct; 7 this week.

  1. This week · 7
  2. When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry
  3. MASKerade: Token-Routed Mask Experts for Dense-to-MoE Upcycling
  4. Reading, Not Manipulating: Leveraging Router Logits for Multimodal Safety in MoE Vision-Language Models also fits: MoE in reinforcement learning and robotics
  5. Memory-Efficient Expert Routing for Distributed MoE Training also fits: Efficient MoE training and inference systems
  6. Distributionally Robust Mixture-of-Experts Training
  7. BRANCH-MoE: Balance-Aware Tree Routing for Large Embedding Models
  8. How Sparse Probability Maps Shape Mixture-of-Experts Routing
  9. Earlier · 20
  10. Frequency Is Not Sensitivity Identifying Safety-Sensitive Experts in Sparse MoE LLM
  11. Conditional Capacity and Routing in Mixture-of-Experts Particle Transformers also fits: MoE theory and gating functions
  12. Dense Mixture-of-Experts as a Reparameterized Wide FFN: A Granularity Sweep at Fixed Compute also fits: MoE theory and gating functions
All 27
  1. SpikeMoE: Brain-Inspired Competitive Routing for Flexible Spiking Mixture-of-Experts also fits: MoE in reinforcement learning and robotics
  2. Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts
  3. Score the Update, Not the Token: Descent-Aligned Routing for Combinatorial LoRA Experts
  4. MoRA: MoE Pruning via Router Bias Learning and Expert Approximation
  5. RAZOR: Pruning Replaceable Experts in LLMs also fits: MoE large language models
  6. Cross-Entropy Guided Routing in Mixture-of-Experts Large Language Models also fits: MoE large language models
  7. Routing in Gradient Space: Balanced Usage Is Not Expert Specialization
  8. MoRE: Scaling mixture of experts with hardware-aware low-rank routing
  9. How to Loop MoE: Flatten the Experts, Untie the Attention
  10. A Persistent State for Auditable Mixture-of-Experts Routing
  11. X-MoD: Practical Scaling Laws for Sparse-Depth Routing Beyond Mixture-of-Depths
  12. EAT: Expert Account Tracker for Efficient MoE Inference also fits: MoE theory and gating functions
  13. Routing Drift Alone Does Not Diagnose Failure in Merged MoE LLMs
  14. HERO-MoE: Historical Expert Routing with Scale-Preserving Fusion
  15. Function Over Form: Distributional Orthogonalization in Mixture-of-Experts with Replica Expert Mechanism also fits: Efficient MoE training and inference systems
  16. IntBMoE: Integrating Block-Level Conditioning into Expert Composition for Full-Participation Mixture-of-Experts
  17. Attention-Aware Routing: Coupling Routing and Attention in MoEs

Foundations

Most cited by papers here

  1. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity2021cited by 192 here
  2. Mixture-of-Experts with Expert Choice Routing2022cited by 88 here
  3. Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts2024cited by 36 here
  4. On the Representation Collapse of Sparse Mixture of Experts2022cited by 33 here
  5. StableMoE: Stable Routing Strategy for Mixture of Experts2022cited by 22 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 193 here
  2. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 143 here
  3. 2022Mixture-of-Experts with Expert Choice Routingcited by 88 here
  4. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 89 here
  5. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 40 here
  6. 2024Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Expertscited by 36 here
  7. 2024DeepSeek-V3 Technical Reportcited by 40 here
17 more it builds on, not in this chain
  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 192 here
  2. 2021Scaling Vision with Sparse Mixture of Expertscited by 45 here
  3. 2021Taming Sparsely Activated Transformer with Stochastic Expertscited by 19 here
  4. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 59 here
  5. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 28 here
  6. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 54 here
  7. 2022StableMoE: Stable Routing Strategy for Mixture of Expertscited by 22 here
  8. 2022On the Representation Collapse of Sparse Mixture of Expertscited by 33 here
  9. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 22 here
  10. 2023From Sparse to Soft Mixtures of Expertscited by 26 here
  11. 2024Mixtral of Expertscited by 61 here
  12. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Modelscited by 34 here
  13. 2024Scaling Laws for Fine-Grained Mixture of Expertscited by 22 here
  14. 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelscited by 17 here
  15. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 50 here
  16. 2024ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routingcited by 18 here
  17. 2025Qwen3 Technical Reportcited by 24 here

The 24 most cited of 530.

Browse

30 of 323, ranked 6–35

  1. 6Taming Sparsely Activated Transformer with Stochastic Experts202119
  2. 7ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routing202418
  3. 8Harder Tasks Need More Experts: Dynamic Routing in MoE Models202414
  4. 9SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification202511
  5. 10Steering MoE LLMs via Expert (De)Activation202511
  6. 11Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Models202410
  7. 12Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy202310
  8. 13Load Balancing Mixture of Experts with Similarity Preserving Routers20259
  9. 14CompeteSMoE -- Effective Training of Sparse Mixture of Experts via Competition20249
  10. 15AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models20249
All 30
  1. 16Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers20239
  2. 17Soft Merging of Experts with Adaptive Routing20239
  3. 18Mixture of A Million Experts20248
  4. 19Statistical Advantages of Perturbing Cosine Router in Mixture of Experts20247
  5. 20MoEUT: Mixture-of-Experts Universal Transformers20247
  6. 21Layerwise Recurrent Router for Mixture-of-Experts20247
  7. 22HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts20237
  8. 23BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts20256
  9. 24Multilingual Routing in Mixture-of-Experts20256
  10. 25Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss20256
  11. 26Towards an empirical understanding of MoE design choices20246
  12. 27Mixture of Attention Heads: Selecting Attention Heads Per Token20226
  13. 28When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models20265
  14. 29Routing Manifold Alignment Improves Generalization of Mixture-of-Experts LLMs20255
  15. 30SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention20235
  16. 31MoEfication: Transformer Feed-forward Layers are Mixtures of Experts20215
  17. 32Understanding Multilingualism in Mixture-of-Experts LLMs: Routing Mechanism, Expert Specialization, and Layerwise Steering20264
  18. 33Probing Semantic Routing in Large Mixture-of-Expert Models20254
  19. 34Domain-Specific Pruning of Large Mixture-of-Experts Models with Few-shot Demonstrations20254
  20. 35Buffer Overflow in Mixture of Experts20244