# Buffer Overflow in Mixture of Experts

arXiv: https://arxiv.org/abs/2402.05526 · HTML: https://mlsanity.com/paper/2402.05526/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts.
- Builds on 8 of its 10 references that are in the map.
- First posted on arXiv 8 Feb 2024.

## Placed in

- Mixture-of-experts / [Expert routing strategies](https://mlsanity.com/moe/expert-routing-strategies/)

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [MegaBlocks: Efficient Sparse Training with Mixture-of-Experts](https://mlsanity.com/paper/2211.15841/) (2022)

## Built on by (most cited first, 13 of 13)

- [LLMmap: Fingerprinting For Large Language Models](https://mlsanity.com/paper/2407.15847/) (2024)
- [Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)](https://mlsanity.com/paper/2407.14937/) (2024)
- [SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification](https://mlsanity.com/paper/2506.17368/) (2025)
- [Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Experts](https://mlsanity.com/paper/2503.05066/) (2025)
- [BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts](https://mlsanity.com/paper/2504.18598/) (2025)
- [Stealing User Prompts from Mixture of Experts](https://mlsanity.com/paper/2410.22884/) (2024)
- [GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs](https://mlsanity.com/paper/2512.21008/) (2025)
- [Large Language Lobotomy: Jailbreaking Mixture-of-Experts via Expert Silencing](https://mlsanity.com/paper/2602.08741/) (2026)
- [Misrouter: Exploiting Routing Mechanisms for Input-Only Attacks on Mixture-of-Experts LLMs](https://mlsanity.com/paper/2605.04446/) (2026)
- [RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress](https://mlsanity.com/paper/2512.23995/) (2025)
- [Quantamination: Dynamic Quantization Leaks Your Data Across the Batch](https://mlsanity.com/paper/2604.26505/) (2026)
- [Multi-Domain Learning with Global Expert Mapping](https://mlsanity.com/paper/2604.18842/) (2026)
- [SparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems via Deterministic Side Channels](https://mlsanity.com/paper/2608.02995/) (2026)

## Abstract (arXiv)

Mixture of Experts (MoE) has become a key ingredient for scaling large foundation models while keeping inference costs steady. We show that expert routing strategies that have cross-batch dependencies are vulnerable to attacks. Malicious queries can be sent to a model and can affect a model's output on other benign queries if they are grouped in the same batch. We demonstrate this via a proof-of-concept attack in a toy experimental setting.