# Trainable Noise Model as an XAI evaluation method: application on Sobol for remote sensing image segmentation

arXiv: https://arxiv.org/abs/2310.01828 · HTML: https://mlsanity.com/paper/2310.01828/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 8 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis](https://mlsanity.com/paper/2111.04138/) (2021)
- [Towards Interpretable Semantic Segmentation via Gradient-weighted Class Activation Mapping](https://mlsanity.com/paper/2002.11434/) (2020)
- [Robust Explainability: A Tutorial on Gradient-Based Attribution Methods for Deep Neural Networks](https://mlsanity.com/paper/2107.11400/) (2021)
- [Towards Better Explanations of Class Activation Mapping](https://mlsanity.com/paper/2102.05228/) (2021)
- [U-Noise: Learnable Noise Masks for Interpretable Image Segmentation](https://mlsanity.com/paper/2101.05791/) (2021)
- [Lebanon Solar Rooftop Potential Assessment using Buildings Segmentation from Aerial Images](https://mlsanity.com/paper/2111.11397/) (2021)

## Built on by (most cited first, 3 of 3)

- [Explainable AI (XAI) in Image Segmentation in Medicine, Industry, and Beyond: A Survey](https://mlsanity.com/paper/2405.01636/) (2024)
- [Deep Sensitivity Analysis for Objective-Oriented Combinatorial Optimization](https://mlsanity.com/paper/2403.00016/) (2024)
- [Entropy-Centric Explainable AI for Remote Sensing Image Segmentation](https://mlsanity.com/paper/2608.11064/) (2026)

## Abstract (arXiv)

eXplainable Artificial Intelligence (XAI) has emerged as an essential requirement when dealing with mission-critical applications, ensuring transparency and interpretability of the employed black box AI models. The significance of XAI spans various domains, from healthcare to finance, where understanding the decision-making process of deep learning algorithms is essential. Most AI-based computer vision models are often black boxes; hence, providing explainability of deep neural networks in image processing is crucial for their wide adoption and deployment in medical image analysis, autonomous driving, and remote sensing applications. Recently, several XAI methods for image classification tasks have been introduced. On the contrary, image segmentation has received comparatively less attention in the context of explainability, although it is a fundamental task in computer vision applications, especially in remote sensing. Only some research proposes gradient-based XAI algorithms for image segmentation. This paper adapts the recent gradient-free Sobol XAI method for semantic segmentation. To measure the performance of the Sobol method for segmentation, we propose a quantitative XAI evaluation method based on a learnable noise model. The main objective of this model is to induce noise on the explanation maps, where higher induced noise signifies low accuracy and vice versa. A benchmark analysis is conducted to evaluate and compare performance of three XAI methods, including Seg-Grad-CAM, Seg-Grad-CAM++ and Seg-Sobol using the proposed noise-based evaluation technique. This constitutes the first attempt to run and evaluate XAI methods using high-resolution satellite images.