# Expert Selections In MoE Models Reveal (Almost) As Much As Text

arXiv: https://arxiv.org/abs/2602.04105 · HTML: https://mlsanity.com/paper/2602.04105/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts.
- Builds on 6 of its 11 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Placed in

- Mixture-of-experts / [Expert routing strategies](https://mlsanity.com/moe/expert-routing-strategies/)

## Builds on (most cited first, 6 of 6)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Information Leakage in Embedding Models](https://mlsanity.com/paper/2004.00053/) (2020)
- [Language Models are Injective and Hence Invertible](https://mlsanity.com/paper/2510.15511/) (2025)
- [MoEcho: Exploiting Side-Channel Attacks to Compromise User Privacy in Mixture-of-Experts LLMs](https://mlsanity.com/paper/2508.15036/) (2025)
- [Universal Zero-shot Embedding Inversion](https://mlsanity.com/paper/2504.00147/) (2025)

## Built on by (most cited first, 6 of 6)

- [The Myth of Expert Specialization in MoEs: Why Routing Reflects Geometry, Not Necessarily Domain Expertise](https://mlsanity.com/paper/2604.09780/) (2026)
- [RouteScan: A Non-Intrusive Approach to Auditing MoE LLMs Safety via Expert Routing Telemetry](https://mlsanity.com/paper/2605.24817/) (2026)
- [Layer-wise MoE Routing Locality under Shared-Prefix Code Generation: Token-Identity Decomposition and Compile-Equivalent Fork Redundancy](https://mlsanity.com/paper/2604.17182/) (2026)
- [Workload Identification with Physical Side Channels for AI Governance](https://mlsanity.com/paper/2609.00309/) (2026)
- [CipherGenome: Homomorphic Inference for Genomic Mixture-of-Experts](https://mlsanity.com/paper/2609.35883/) (2026)
- [When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry](https://mlsanity.com/paper/2610.10616/) (2026)

## Abstract (arXiv)

We present a text-reconstruction attack on mixture-of-experts (MoE) language models that recovers tokens from expert selections alone. In MoE models, each token is routed to a subset of expert subnetworks; we show these routing decisions leak substantially more information than previously understood. Prior work using logistic regression achieves limited reconstruction; we show that a 3-layer MLP improves this to 63.1% top-1 accuracy, and that a transformer-based sequence decoder recovers 91.2% of tokens top-1 (94.8% top-10) on 32-token sequences from OpenWebText after training on 100M tokens. These results connect MoE routing to the broader literature on embedding inversion. We outline practical leakage scenarios (e.g., distributed inference and side channels) and show that adding noise reduces but does not eliminate reconstruction. Our findings suggest that expert selections in MoE deployments should be treated as sensitive as the underlying text.