# Expert Routing for Communication-Efficient MoE via Finite Expert Banks

arXiv: https://arxiv.org/abs/2605.05278 · HTML: https://mlsanity.com/paper/2605.05278/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts.
- Builds on 11 of its 25 references that are in the map.
- First posted on arXiv 6 May 2026.

## Placed in

- Mixture-of-experts / [Expert routing strategies](https://mlsanity.com/moe/expert-routing-strategies/)

## Builds on (most cited first, 11 of 11)

- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Information-theoretic analysis of generalization capability of learning algorithms](https://mlsanity.com/paper/1705.07809/) (2017)
- [Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates](https://mlsanity.com/paper/1911.02151/) (2019)
- [Tightening Mutual Information Based Bounds on Generalization Error](https://mlsanity.com/paper/1901.04609/) (2019)
- [Generalization Error Bounds for Noisy, Iterative Algorithms](https://mlsanity.com/paper/1801.04295/) (2018)
- [Accessing From The Sky: A Tutorial on UAV Communications for 5G and Beyond](https://mlsanity.com/paper/1903.05289/) (2019)
- [Hierarchically Structured Task-Agnostic Continual Learning](https://mlsanity.com/paper/2211.07725/) (2022)
- [Tighter Risk Bounds for Mixtures of Experts](https://mlsanity.com/paper/2410.10397/) (2024)
- [Mixture-of-Experts under Finite-Rate Gating: Communication--Generalization Trade-offs](https://mlsanity.com/paper/2602.15091/) (2026)
- [Learning-Augmented Perfectly Secure Collaborative Matrix Multiplication](https://mlsanity.com/paper/2601.09916/) (2026)

## Built on by (most cited first, 1 of 1)

- [Sparse In-Network Learning via Shortest-Path Backpropagation and Finite-Rate Gating](https://mlsanity.com/paper/2605.23424/) (2026)

## Abstract (arXiv)

Resource-efficient machine learning increasingly uses sparse Mixture-of-Experts (MoE) architectures, where the gate acts as both a learning component and a routing interface controlling computation, communication, and accuracy. Motivated by finite-rate interpretations of MoE gating, we treat the gate as a stochastic channel and use I(X;T) to quantify the routing information available to the selected expert. To make the associated information quantities tractable beyond synthetic examples, we develop a finite-bank MNIST construction using pretrained CNN experts and a discrete, data-dependent selection rule. Since the selected model belongs to a finite candidate set, the algorithmic mutual information I(S;W) admits a closed-form discrete-entropy estimator from the empirical posterior q(W|S). Sweeping a data-dependence parameter α, we observe that widehat I(S;W) monotonically tracks the generalization gap, while the Xu-Raginsky bound exhibits the expected looseness. We also compare with a uniform union-bound baseline and introduce an empirical estimator of I(X;T) together with a Blahut-Arimoto procedure for tracing an accuracy-rate curve over the expert bank. The proposed framework provides a practical tool for analyzing resource-aware MoE inference systems and for interpreting I(X;T) and D(Rg) as design proxies for efficient expert routing.