mlsanityData to 14 Oct 2026

Routing Sensitivity Without Controllability: A Diagnostic Study of Fairness in MoE Language Models

arXiv 2603.27141 ↗First posted 28 Mar 2026cs.CLcited by 1

On the map: Mixture-of-experts → Expert routing strategies

Abstract (arXiv)

Demographic prompts are routed differently from neutral prompts in Mixture-of-Experts (MoE) language models, motivating tests of routing-level stereotype control. We introduce Fairness-Aware Routing Equilibrium (FARE), a diagnostic framework combining demographic routing profiles, empirical layer selection, and fixed inference-time reweighting, and evaluate five MoE architectures in English. At the selected operating points, CrowS-Pairs preference changes by at most 1.3 percentage points; DeepSeekMoE selects no intervention. Paired 95% confidence intervals exclude decreases larger than 2.2 points on each intervened model, and the only nominally significant change (Qwen1.5, p = 0.015) does not survive multiple-comparison correction. OLMoE and Qwen3 nevertheless change nearly every top-k expert set. Noise controls, random and truncated synthetic profiles, and hard masking also move preference by at most 1.5 points. Four generation protocols on OLMoE, Mixtral, Qwen1.5, and Qwen3 show no consistent change in the measured toxicity, lexical, or reference-overlap metrics. Selection and evaluation items overlap, so these comparisons are not independent evaluations. The tested reweighting procedure offers limited stereotype control.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Expert routing strategies, one of 12 directions in Mixture-of-experts. Builds on 22 of its 26 references that are in the map. First posted on arXiv 28 Mar 2026.

Builds on

22 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026MoE Lens -- An Expert Is All You Need6 Mar
  2. 2025SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification20 Jun
  3. 2024Towards an empirical understanding of MoE design choices20 Feb
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity11 Jan

Other references · 18

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  6. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  7. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  8. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  9. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  10. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 301
  11. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  12. 2024MaxMin-RLHF: Alignment with Diverse Human Preferencescited by 123
6 more
  1. 2024Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypescited by 59
  2. 2025FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steeringcited by 40
  3. 2024Prompting Techniques for Reducing Social Bias in LLMs through System 1 and System 2 Cognitive Processescited by 39
  4. 2025Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Trainingcited by 16
  5. 2025Shifting Perspectives: Steering Vectors for Robust Bias Mitigation in LLMscited by 14
  6. 2025R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Expertscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026THESIS-MoE: Trainable Hierarchical Extraction and SteerIng of Sycophancy in Mixture-of-Expertscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MoE Routing Testbed: Studying Expert Specialization and Routing Behavior at Small Scale
  2. 2026Hierarchical Mixture-of-Experts with Two-Stage Optimization
  3. 2026When Are Experts Misrouted? Counterfactual Routing Analysis in Mixture-of-Experts Language Models
  4. 2025Parameter-Efficient Routed Fine-Tuning: Mixture-of-Experts Demands Mixture of Adaptation Modules
  5. 2026Mixture of Experts for Low-Resource LLMs
  6. 2025Multilingual Routing in Mixture-of-Experts
  7. 2026Safety-Oriented Routing Analysis of Mixtral MoE Under Benign and Harmful Prompts
  8. 2026Equifinality in Mixture of Experts: Routing Topology Does Not Determine Language Modeling Quality