mlsanityData to 7 Oct 2026

Steering MoE LLMs via Expert (De)Activation

arXiv 2509.09660 ↗First posted 11 Sep 2025cs.CL cs.LGcited by 23

On the map: Mixture-of-experts → Expert routing strategies

also in 1 more
Abstract (arXiv)

Mixture-of-Experts (MoE) in Large Language Models (LLMs) routes each token through a subset of specialized Feed-Forward Networks (FFN), known as experts. We present SteerMoE, a framework to steer MoE models by detecting and controlling behavior-associated experts. We detect key experts by comparing how often they activate between paired inputs that demonstrate opposite behaviors (e.g., safe vs. unsafe). By selectively activating or deactivating such experts during inference, we control behaviors like faithfulness and safety without fine-tuning. Across 11 benchmarks and 6 LLMs, our steering raises safety by up to +20% and faithfulness by +27%. Alternatively, unsafe steering drops safety by -41% alone, and -100% when combined with existing jailbreak methods, bypassing all safety guardrails. Overall, SteerMoE offers a lightweight, effective, and widely applicable test-time control, while revealing unique vulnerabilities in MoE LLMs. https://github.com/adobe-research/SteerMoE

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts. Builds on 38 of its 46 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

38 of 46 references.

References · 38

  1. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  7. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  11. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  12. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
26 more
  1. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  2. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  3. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
  4. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 272
  5. 2024A Survey on Mixture of Experts in Large Language Modelscited by 233
  6. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  7. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  8. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Modelscited by 185
  9. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  10. 2020A Diagnostic Study of Explainability Techniques for Text Classificationcited by 158
  11. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  12. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 85
  13. 2023Inseq: An Interpretability Toolkit for Sequence Generation Modelscited by 78
  14. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  15. 2023Word Embeddings Are Steers for Language Modelscited by 70
  16. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineeringcited by 60
  17. 2024A Closer Look into Mixture-of-Experts in Large Language Modelscited by 47
  18. 2022GlobEnc: Quantifying Global Token Attribution by Incorporating the Whole Encoder Layer in Transformerscited by 44
  19. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  20. 2023Fake Alignment: Are LLMs Really Aligned Well?cited by 39
  21. 2025Real-Time Detection of Hallucinated Entities in Long-Form Generationcited by 33
  22. 2023DecompX: Explaining Transformers Decisions by Propagating Token Decompositioncited by 31
  23. 2024Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attackscited by 17
  24. 2025Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Trainingcited by 16
  25. 2025The Blessing and Curse of Dimensionality in Safety Alignmentcited by 9
  26. 2023Hallucination Augmented Recitations for Language Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 19.

23 papers.

Sort:

By area:

  1. 2025Multilingual Routing in Mixture-of-Expertscited by 25
  2. 2025EasySteer: A Unified Framework for High-Performance and Extensible LLM Steeringcited by 13
  3. 2025Defending MoE LLMs against Harmful Fine-Tuning via Safety Routing Alignmentcited by 8
  4. 2026Seeing but Not Thinking: Routing Distraction in Multimodal Mixture-of-Expertscited by 8
  5. 2026Sparse Models, Sparse Safety: Unsafe Routes in Mixture-of-Experts LLMscited by 6
  6. 2026Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistencycited by 6
  7. 2026RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMscited by 4
  8. 2026What Gets Activated: Uncovering Domain and Driver Experts in MoE Language Modelscited by 3
  9. 2026RASA: Routing-Aware Safety Alignment for Mixture-of-Experts Modelscited by 3
  10. 2026Probing Persona-Dependent Preferences in Language Modelscited by 3
  11. 2026Extracting Small Translation Specialists from LLMs by Aggressively Pruning Expertscited by 3
  12. 2026Sparse Mixture-of-Experts Reward Models Learn Interpretable and Specialized Experts for Personalized Preference Modelingcited by 2
11 more
  1. 2025ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attackcited by 1
  2. 2026Residual Paving: Diagnosing the Routing Bottleneck in Selective Refusal Editingcited by 0
  3. 2026MESA: Improving MoE Safety Alignment via Decentralized Expertisecited by 0
  4. 2026Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Modelscited by 0
  5. 2026Decomposing Refusal Steering in Mixture-of-Experts Modelscited by 0
  6. 2026LoMC: Localized Multidirectional Correction for Refusal Suppression in Routed Foundation Modelscited by 0
  7. 2026xMIx: High-Performance Serving-Time Platform for Mechanistic Interpretability Appscited by 0
  8. 2026RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Modelscited by 0
  9. 2026SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignmentcited by 0
  10. 2026From Routing Signals to Selective Review: Visual regrounding in MoE VLMscited by 0
  11. 2026Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safety
  2. 2026Do Domain-specific Experts exist in MoE-based LLMs?
  3. 2025GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs
  4. 2025BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts
  5. 2025Ban&Pick: Enhancing Performance and Efficiency of MoE-LLMs via Smarter Routing
  6. 2025Exploiting the Experts: Unauthorized Compression in MoE-LLMs
  7. 2026RASET: Router-Agnostic Safety-Critical Expert Tuning Exposes Localized Safety Enforcement Failures in Mixture-of-Experts LLMs
  8. 2025SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification