# MiSuRe is all you need to explain your image segmentation

arXiv: https://arxiv.org/abs/2406.12173 · HTML: https://mlsanity.com/paper/2406.12173/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [RISE: Randomized Input Sampling for Explanation of Black-box Models](https://mlsanity.com/paper/1806.07421/) (2018)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1910.01279/) (2019)
- [Towards Interpretable Semantic Segmentation via Gradient-weighted Class Activation Mapping](https://mlsanity.com/paper/2002.11434/) (2020)
- [Grid Saliency for Context Explanations of Semantic Segmentation](https://mlsanity.com/paper/1907.13054/) (2019)
- [Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanations](https://mlsanity.com/paper/2211.11426/) (2022)
- [The Intrinsic Manifolds of Radiological Images and their Role in Deep Learning](https://mlsanity.com/paper/2207.02797/) (2022)
- [U-Noise: Learnable Noise Masks for Interpretable Image Segmentation](https://mlsanity.com/paper/2101.05791/) (2021)
- [SegNBDT: Visual Decision Rules for Segmentation](https://mlsanity.com/paper/2006.06868/) (2020)
- [Is the U-Net Directional-Relationship Aware?](https://mlsanity.com/paper/2207.02574/) (2022)

## Built on by (most cited first, 3 of 3)

- [Here Comes the Explanation: A Shapley Perspective on Multi-contrast Medical Image Segmentation](https://mlsanity.com/paper/2504.04645/) (2025)
- [Leveraging Causal Reasoning Method for Explaining Medical Image Segmentation Models](https://mlsanity.com/paper/2602.20511/) (2026)
- [Efficient KernelSHAP Explanations for Patch-based 3D Medical Image Segmentation](https://mlsanity.com/paper/2604.11775/) (2026)

## Abstract (arXiv)

The last decade of computer vision has been dominated by Deep Learning architectures, thanks to their unparalleled success. Their performance, however, often comes at the cost of explainability owing to their highly non-linear nature. Consequently, a parallel field of eXplainable Artificial Intelligence (XAI) has developed with the aim of generating insights regarding the decision making process of deep learning models. An important problem in XAI is that of the generation of saliency maps. These are regions in an input image which contributed most towards the model's final decision. Most work in this regard, however, has been focused on image classification, and image segmentation - despite being a ubiquitous task - has not received the same attention. In the present work, we propose MiSuRe (Minimally Sufficient Region) as an algorithm to generate saliency maps for image segmentation. The goal of the saliency maps generated by MiSuRe is to get rid of irrelevant regions, and only highlight those regions in the input image which are crucial to the image segmentation decision. We perform our analysis on 3 datasets: Triangle (artificially constructed), COCO-2017 (natural images), and the Synapse multi-organ (medical images). Additionally, we identify a potential usecase of these post-hoc saliency maps in order to perform post-hoc reliability of the segmentation model.