mlsanityData to 14 Oct 2026

Routing-Free Mixture-of-Experts

arXiv 2604.00801 ↗First posted 1 Apr 2026cs.LG cs.AI cs.CLcited by 2

On the map: Mixture-of-experts → Expert routing strategies

Abstract (arXiv)

Standard Mixture-of-Experts (MoE) models rely on centralized routing mechanisms that introduce rigid inductive biases. We propose Routing-Free MoE which eliminates any hard-coded centralized designs including external routers, Softmax, Top-K and load balancing, instead encapsulating all activation functionalities within individual experts and directly optimized through continuous gradient flow, enabling each expert to determine its activation entirely on its own. We introduce a unified adaptive load-balancing framework to simultaneously optimize both expert-balancing and token-balancing objectives through a configurable interpolation, allowing flexible and customizable resource allocation. Extensive experiments show that Routing-Free MoE can consistently outperform baselines with better scalability and robustness. We analyze its behavior in detail and offer insights that may facilitate future MoE design ad optimization.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts. Builds on 21 of its 62 references that are in the map. First posted on arXiv 1 Apr 2026.

Builds on

21 of 62 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules4 Aug
  2. 2025Load Balancing Mixture of Experts with Similarity Preserving Routers16 Jun
  3. 2024Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Experts28 Aug

Other references · 18

  1. 2020Scaling Laws for Neural Language Modelscited by 6,586
  2. 2024Mixtral of Expertscited by 1,823
  3. 2020GLU Variants Improve Transformercited by 1,804
  4. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  5. 2024DeepSeek-V3 Technical Reportcited by 811
  6. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  7. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 278
  8. 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpointscited by 209
  9. 2023Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuningcited by 106
  10. 2024Higher Layers Need More LoRA Expertscited by 102
  11. 2024Dense Training, Sparse Inference: Rethinking Training of Mixture-of-Experts Language Modelscited by 42
  12. 2024ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Schedulingcited by 19
6 more
  1. 2025LLM-Based Routing in Mixture of Experts: A Novel Framework for Tradingcited by 7
  2. 2025Autonomy-of-Experts Modelscited by 7
  3. 2025Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokenscited by 5
  4. 2024On the Role of Discrete Representation in Sparse Mixture of Expertscited by 4
  5. 2025Rethinking Sparse Mixture of Experts from a Unified Perspectivecited by 1
  6. 2025Fine-grained MoE Load Balancing with Linear Programmingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Detecting a Route Flip Is Easier Than Knowing Whether to Fix It: Causal Route-Mediated Damage in Quantized Mixture-of-Expertscited by 2
  2. 2026Raw-Routed Mixture of Adapters: A Causal Intervention for Routing Collapse in Time Series Foundation Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Hierarchical Mixture-of-Experts with Two-Stage Optimization
  2. 2026Beyond Routing: Decoupling Expert Dispatch and Aggregation in Sparse Mixture-of-Experts
  3. 2026Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation
  4. 2022SMILE: Scaling Mixture-of-Experts with Efficient Bi-level Routing
  5. 2026IntBMoE: Integrating Block-Level Conditioning into Expert Composition for Full-Participation Mixture-of-Experts
  6. 2026Three Phases of Expert Routing: How Load Balance Evolves During Mixture-of-Experts Training
  7. 2026Distributionally Robust Mixture-of-Experts Training
  8. 2025Maximum Score Routing For Mixture-of-Experts