# Weakly Supervised Pixel-Level Annotation with Visual Interpretability

arXiv: https://arxiv.org/abs/2502.17824 · HTML: https://mlsanity.com/paper/2502.17824/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 54 references that are in the map.
- First posted on arXiv 25 Feb 2025.

## Builds on (most cited first, 25 of 25)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [GroupViT: Semantic Segmentation Emerges from Text Supervision](https://mlsanity.com/paper/2202.11094/) (2022)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems](https://mlsanity.com/paper/1907.10456/) (2019)
- [Multi-class Token Transformer for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2203.02891/) (2022)
- [Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis](https://mlsanity.com/paper/2210.03736/) (2022)
- [Visual Feature Attribution using Wasserstein GANs](https://mlsanity.com/paper/1711.08998/) (2017)
- [A comprehensive survey on deep active learning in medical image analysis](https://mlsanity.com/paper/2310.14230/) (2023)
- [An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection](https://mlsanity.com/paper/1810.10850/) (2018)
- [Uncertainty-inspired Open Set Learning for Retinal Anomaly Identification](https://mlsanity.com/paper/2304.03981/) (2023)
- [MagGAN: High-Resolution Face Attribute Editing with Mask-Guided Generative Adversarial Network](https://mlsanity.com/paper/2010.01424/) (2020)
- [Developing a Fidelity Evaluation Approach for Interpretable Machine Learning](https://mlsanity.com/paper/2106.08492/) (2021)
- [Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box](https://mlsanity.com/paper/2203.02399/) (2022)
- [Shifting to Machine Supervision: Annotation-Efficient Semi and Self-Supervised Learning for Automatic Medical Image Segmentation and Classification](https://mlsanity.com/paper/2311.10319/) (2023)
- [Weakly-supervised segmentation using inherently-explainable classification models and their application to brain tumour classification](https://mlsanity.com/paper/2206.05148/) (2022)
- [Semi-Supervised and Self-Supervised Collaborative Learning for Prostate 3D MR Image Segmentation](https://mlsanity.com/paper/2211.08840/) (2022)
- [MDF-Net for abnormality detection by fusing X-rays with clinical data](https://mlsanity.com/paper/2302.13390/) (2023)
- [ReFit: A Framework for Refinement of Weakly Supervised Semantic Segmentation using Object Border Fitting for Medical Images](https://mlsanity.com/paper/2303.07853/) (2023)

## Abstract (arXiv)

Medical image annotation is essential for diagnosing diseases, yet manual annotation is time-consuming, costly, and prone to variability among experts. To address these challenges, we propose an automated explainable annotation system that integrates ensemble learning, visual explainability, and uncertainty quantification. Our approach combines three pre-trained deep learning models - ResNet50, EfficientNet, and DenseNet - enhanced with XGrad-CAM for visual explanations and Monte Carlo Dropout for uncertainty quantification. This ensemble mimics the consensus of multiple radiologists by intersecting saliency maps from models that agree on the diagnosis while uncertain predictions are flagged for human review. We evaluated our system using the TBX11K medical imaging dataset and a Fire segmentation dataset, demonstrating its robustness across different domains. Experimental results show that our method outperforms baseline models, achieving 93.04% accuracy on TBX11K and 96.4% accuracy on the Fire dataset. Moreover, our model produces precise pixel-level annotations despite being trained with only image-level labels, achieving Intersection over Union IoU scores of 36.07% and 64.7%, respectively. By enhancing the accuracy and interpretability of image annotations, our approach offers a reliable and transparent solution for medical diagnostics and other image analysis tasks.