mlsanityData to 14 Oct 2026

Buffer Overflow in Mixture of Experts

arXiv 2402.05526 ↗First posted 8 Feb 2024cs.CR cs.LGcited by 13

On the map: Mixture-of-experts → Expert routing strategies

Abstract (arXiv)

Mixture of Experts (MoE) has become a key ingredient for scaling large foundation models while keeping inference costs steady. We show that expert routing strategies that have cross-batch dependencies are vulnerable to attacks. Malicious queries can be sent to a model and can affect a model's output on other benign queries if they are grouped in the same batch. We demonstrate this via a proof-of-concept attack in a toy experimental setting.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts. Builds on 8 of its 10 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

8 of 10 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity11 Jan

Other references · 7

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  3. 2024Mixtral of Expertscited by 1,823
  4. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  5. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  6. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  7. 2022MegaBlocks: Efficient Sparse Training with Mixture-of-Expertscited by 209

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 5.

13 papers.

Sort:

By area:

  1. 2024LLMmap: Fingerprinting For Large Language Modelscited by 58
  2. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 46
  3. 2025SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identificationcited by 18
  4. 2025Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Expertscited by 15
  5. 2025BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Expertscited by 14
  6. 2024Stealing User Prompts from Mixture of Expertscited by 12
  7. 2025GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMscited by 11
  8. 2026Large Language Lobotomy: Jailbreaking Mixture-of-Experts via Expert Silencingcited by 4
  9. 2026Misrouter: Exploiting Routing Mechanisms for Input-Only Attacks on Mixture-of-Experts LLMscited by 3
  10. 2025RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stresscited by 2
  11. 2026Quantamination: Dynamic Quantization Leaks Your Data Across the Batchcited by 2
  12. 2026Multi-Domain Learning with Global Expert Mappingcited by 0
1 more
  1. 2026SparSEEty: Extracting Tokens from Sparsity-Exploiting LLM Serving Systems via Deterministic Side Channelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Towards a Statistical Understanding of Mixture-of-Experts
  2. 2026MoE Routing Testbed: Studying Expert Specialization and Routing Behavior at Small Scale
  3. 2025Bayesian Mixture-of-Experts: Towards Making LLMs Know What They Don't Know
  4. 2025Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts
  5. 2026Hierarchical Mixture-of-Experts with Two-Stage Optimization
  6. 2026RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs
  7. 2025Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models
  8. 2026Share First, Route What Remains: A Unified Framework for Token-Adaptive MoE Computation