# Towards Adversarial Robustness of Model-Level Mixture-of-Experts Architectures for Semantic Segmentation

arXiv: https://arxiv.org/abs/2412.11608 · HTML: https://mlsanity.com/paper/2412.11608/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 34 of its 44 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale](https://mlsanity.com/paper/2201.05596/) (2022)
- [Adversarial Risk and the Dangers of Evaluating Against Weak Attacks](https://mlsanity.com/paper/1802.05666/) (2018)
- [Improving Adversarial Robustness via Promoting Ensemble Diversity](https://mlsanity.com/paper/1901.08846/) (2019)
- [A2D2: Audi Autonomous Driving Dataset](https://mlsanity.com/paper/2004.06320/) (2020)
- [On the Robustness of Semantic Segmentation Models to Adversarial Attacks](https://mlsanity.com/paper/1711.09856/) (2017)
- [Universal Adversarial Perturbations Against Semantic Image Segmentation](https://mlsanity.com/paper/1704.05712/) (2017)
- [Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes](https://mlsanity.com/paper/1611.08323/) (2016)
- [Improving Adversarial Robustness of Ensembles with Diversity Training](https://mlsanity.com/paper/1901.09981/) (2019)
- [Network of Experts for Large-Scale Image Categorization](https://mlsanity.com/paper/1604.06119/) (2016)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Adversarial Examples for Semantic Image Segmentation](https://mlsanity.com/paper/1703.01101/) (2017)
- [Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networks](https://mlsanity.com/paper/1709.03423/) (2017)
- [Robust Mixture-of-Expert Training for Convolutional Neural Networks](https://mlsanity.com/paper/2308.10110/) (2023)
- [On the Adversarial Robustness of Mixture of Experts](https://mlsanity.com/paper/2210.10253/) (2022)
- [Robustness to Adversarial Examples through an Ensemble of Specialists](https://mlsanity.com/paper/1702.06856/) (2017)
- [Adversarial Examples on Segmentation Models Can be Easy to Transfer](https://mlsanity.com/paper/2111.11368/) (2021)

## Built on by (most cited first, 5 of 5)

- [A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications](https://mlsanity.com/paper/2503.07137/) (2025)
- [Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentation](https://mlsanity.com/paper/2509.04816/) (2025)
- [Divided We Fall: Defending Against Adversarial Attacks via Soft-Gated Fractional Mixture-of-Experts with Randomized Adversarial Training](https://mlsanity.com/paper/2512.20821/) (2025)
- [MME: Mixture of Mesh Experts with Random Walk Transformer Gating](https://mlsanity.com/paper/2603.00828/) (2026)
- [RoME: Robust Mixture of Low-Rank Experts against Multiple Adversarial Perturbations](https://mlsanity.com/paper/2607.06109/) (2026)

## Abstract (arXiv)

Vulnerability to adversarial attacks is a well-known deficiency of deep neural networks. Larger networks are generally more robust, and ensembling is one method to increase adversarial robustness: each model's weaknesses are compensated by the strengths of others. While an ensemble uses a deterministic rule to combine model outputs, a mixture of experts (MoE) includes an additional learnable gating component that predicts weights for the outputs of the expert models, thus determining their contributions to the final prediction. MoEs have been shown to outperform ensembles on specific tasks, yet their susceptibility to adversarial attacks has not been studied yet. In this work, we evaluate the adversarial vulnerability of MoEs for semantic segmentation of urban and highway traffic scenes. We show that MoEs are, in most cases, more robust to per-instance and universal white-box adversarial attacks and can better withstand transfer attacks. Our code is available at urlhttps://github.com/KASTEL-MobilityLab/mixtures-of-experts/.