mlsanityData to 14 Oct 2026

Explainable AI (XAI) in Image Segmentation in Medicine, Industry, and Beyond: A Survey

arXiv 2405.01636 ↗First posted 2 May 2024cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Artificial Intelligence (XAI) has found numerous applications in computer vision. While image classification-based explainability techniques have garnered significant attention, their counterparts in semantic segmentation have been relatively neglected. Given the prevalent use of image segmentation, ranging from medical to industrial deployments, these techniques warrant a systematic look. In this paper, we present the first comprehensive survey on XAI in semantic image segmentation. This work focuses on techniques that were either specifically introduced for dense prediction tasks or were extended for them by modifying existing methods in classification. We analyze and categorize the literature based on application categories and domains, as well as the evaluation metrics and datasets used. We also propose a taxonomy for interpretable semantic segmentation, and discuss potential challenges and future research directions.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 110 of its 215 references that are in the map. First posted on arXiv 2 May 2024.

Builds on

110 of 215 references.

References · 110

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  5. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  8. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  9. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  10. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  11. 2013Visualizing and Understanding Convolutional Networkscited by 2,290
  12. 2016Concrete Problems in AI Safetycited by 1,642
38 more
  1. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
  2. 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,552
  3. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  4. 2019Natural Adversarial Examplescited by 1,408
  5. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  6. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  7. 2018Sanity Checks for Saliency Mapscited by 1,156
  8. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  9. 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,089
  10. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  11. 2014Striving for Simplicity: The All Convolutional Netcited by 990
  12. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  13. 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 838
  14. 2018This Looks Like That: Deep Learning for Interpretable Image Recognitioncited by 737
  15. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  16. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 669
  17. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  18. 2021The Medical Segmentation Decathloncited by 628
  19. 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 589
  20. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  21. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 509
  22. 2019A large annotated medical image dataset for the development and evaluation of segmentation algorithmscited by 373
  23. 2017The (Un)reliability of saliency methodscited by 333
  24. 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 314
  25. 2019On the (In)fidelity and Sensitivity for Explanationscited by 310
  26. 2020CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentationcited by 309
  27. 2020A Survey on Neural Network Interpretabilitycited by 291
  28. 2017Adversarial Examples for Semantic Segmentation and Object Detectioncited by 275
  29. 2018Metrics for Explainable AI: Challenges and Prospectscited by 260
  30. 2019REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographscited by 228
  31. 2019BCN20000: Dermoscopic Lesions in the Wildcited by 225
  32. 2016A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Imagescited by 221
  33. 2019Explanations can be manipulated and geometry is to blamecited by 216
  34. 2020A2D2: Audi Autonomous Driving Datasetcited by 190
  35. 2019Explainable Machine Learning for Scientific Insights and Discoveriescited by 166
  36. 2016Multivariate mixture model for myocardium segmentation combining multi-source imagescited by 133
  37. 2019Deep Semantic Segmentation of Natural and Medical Images: A Reviewcited by 125
  38. 2021Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypescited by 120

50 most cited of 110 All 110 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2025A Survey on Interpretability in Visual Recognitioncited by 2
  2. 2025An interpretable approach to automating the assessment of biofouling in video footagecited by 1
  3. 2025A Quantitative Evaluation Framework for Explainable AI in Semantic Segmentationcited by 1
  4. 2026Interpretable Vision Transformers in Monocular Depth Estimation via SVDAcited by 1
  5. 2025EU-Nets: Enhanced, Explainable and Parsimonious U-Netscited by 0
  6. 2026Leveraging Causal Reasoning Method for Explaining Medical Image Segmentation Modelscited by 0
  7. 2026Efficient KernelSHAP Explanations for Patch-based 3D Medical Image Segmentationcited by 0
  8. 2026Evaluating the Alignment Between GeoAI Explanations and Domain Knowledge in Satellite-Based Flood Mappingcited by 0
  9. 2026Evaluation Cards for XAI Metricscited by 0
  10. 2026Self-Explainability in Self-Adaptive and Self-Organising Systems: Status and Research Directionscited by 0
  11. 2026A Quantitative Evaluation Framework for Temporal Explainability in Echocardiographic Video Segmentationcited by 0