# Uncertainty-weighted Loss Functions for Improved Adversarial Attacks on Semantic Segmentation

arXiv: https://arxiv.org/abs/2310.17436 · HTML: https://mlsanity.com/paper/2310.17436/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 34 references that are in the map.
- First posted on arXiv 26 Oct 2023.

## Builds on (most cited first, 30 of 30)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [On the Robustness of Semantic Segmentation Models to Adversarial Attacks](https://mlsanity.com/paper/1711.09856/) (2017)
- [Universal Adversarial Perturbations Against Semantic Image Segmentation](https://mlsanity.com/paper/1704.05712/) (2017)
- [SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustness](https://mlsanity.com/paper/2207.12391/) (2022)
- [Evaluating the Robustness of Semantic Segmentation for Autonomous Driving against Real-World Adversarial Patch Attacks](https://mlsanity.com/paper/2108.06179/) (2021)
- [Houdini: Fooling Deep Structured Prediction Models](https://mlsanity.com/paper/1707.05373/) (2017)
- [Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation](https://mlsanity.com/paper/1810.05162/) (2018)
- [Dynamic Divide-and-Conquer Adversarial Training for Robust Semantic Segmentation](https://mlsanity.com/paper/2003.06555/) (2020)
- [Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent](https://mlsanity.com/paper/2106.15023/) (2021)
- [CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks](https://mlsanity.com/paper/2302.02213/) (2023)
- [Improved Noise and Attack Robustness for Semantic Segmentation by Using Multi-Task Training with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2004.11072/) (2020)
- [Indirect Local Attacks for Context-aware Semantic Segmentation Networks](https://mlsanity.com/paper/1911.13038/) (2019)
- [Proximal Splitting Adversarial Attacks for Semantic Segmentation](https://mlsanity.com/paper/2206.07179/) (2022)
- [Time-Dynamic Estimates of the Reliability of Deep Semantic Segmentation Networks](https://mlsanity.com/paper/1911.05075/) (2019)
- [Certified Defences Against Adversarial Patch Attacks on Semantic Segmentation](https://mlsanity.com/paper/2209.05980/) (2022)
- [DAPAS : Denoising Autoencoder to Prevent Adversarial attack in Semantic Segmentation](https://mlsanity.com/paper/1908.05195/) (2019)
- [The Vulnerability of Semantic Segmentation Networks to Adversarial Attacks in Autonomous Driving: Enhancing Extensive Environment Sensing](https://mlsanity.com/paper/2101.03924/) (2021)
- [Pixel-wise Gradient Uncertainty for Convolutional Neural Networks applied to Out-of-Distribution Segmentation](https://mlsanity.com/paper/2303.06920/) (2023)
- [Semantically Stealthy Adversarial Attacks against Segmentation Models](https://mlsanity.com/paper/2104.01732/) (2021)
- [A Certified Radius-Guided Attack Framework to Image Segmentation Models](https://mlsanity.com/paper/2304.02693/) (2023)
- [Uncertainty-based Detection of Adversarial Attacks in Semantic Segmentation](https://mlsanity.com/paper/2305.12825/) (2023)

## Built on by (most cited first, 9 of 9)

- [Reducing Texture Bias of Deep Neural Networks via Edge Enhancing Diffusion](https://mlsanity.com/paper/2402.09530/) (2024)
- [SegTrans: Transferable Adversarial Examples for Segmentation Models](https://mlsanity.com/paper/2510.08922/) (2025)
- [Uni-SLAM: Uncertainty-Aware Neural Implicit SLAM for Real-Time Dense Indoor Scene Reconstruction](https://mlsanity.com/paper/2412.00242/) (2024)
- [Semantic Representation Attack against Aligned Large Language Models](https://mlsanity.com/paper/2509.19360/) (2025)
- [Bridging the Task Gap: Multi-Task Adversarial Transferability in CLIP and Its Derivatives](https://mlsanity.com/paper/2509.23917/) (2025)
- [Detecting Adversarial Attacks in Semantic Segmentation via Uncertainty Estimation: A Deep Analysis](https://mlsanity.com/paper/2408.10021/) (2024)
- [Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networks](https://mlsanity.com/paper/2507.10239/) (2025)
- [Proxy-Embedding as an Adversarial Teacher: An Embedding-Guided Bidirectional Attack for Referring Expression Segmentation Models](https://mlsanity.com/paper/2506.16157/) (2025)
- [IGME: Efficient Chained Method Ensemble for Transferable Semantic Segmentation Attacks](https://mlsanity.com/paper/2607.27465/) (2026)

## Abstract (arXiv)

State-of-the-art deep neural networks have been shown to be extremely powerful in a variety of perceptual tasks like semantic segmentation. However, these networks are vulnerable to adversarial perturbations of the input which are imperceptible for humans but lead to incorrect predictions. Treating image segmentation as a sum of pixel-wise classifications, adversarial attacks developed for classification models were shown to be applicable to segmentation models as well. In this work, we present simple uncertainty-based weighting schemes for the loss functions of such attacks that (i) put higher weights on pixel classifications which can more easily perturbed and (ii) zero-out the pixel-wise losses corresponding to those pixels that are already confidently misclassified. The weighting schemes can be easily integrated into the loss function of a range of well-known adversarial attackers with minimal additional computational overhead, but lead to significant improved perturbation performance, as we demonstrate in our empirical analysis on several datasets and models.