# Explanations of Classifiers Enhance Medical Image Segmentation via End-to-end Pre-training

arXiv: https://arxiv.org/abs/2401.08469 · HTML: https://mlsanity.com/paper/2401.08469/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 16 Jan 2024.

## Builds on (most cited first, 21 of 21)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Large-scale weakly-supervised pre-training for video action recognition](https://mlsanity.com/paper/1905.00561/) (2019)
- [Revisiting Weakly Supervised Pre-Training of Visual Perception Models](https://mlsanity.com/paper/2201.08371/) (2022)
- [Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs By Comparing Image Representations](https://mlsanity.com/paper/2007.07423/) (2020)
- [COVID-19 Infection Localization and Severity Grading from Chest X-ray Images](https://mlsanity.com/paper/2103.07985/) (2021)
- [PixMIM: Rethinking Pixel Reconstruction in Masked Image Modeling](https://mlsanity.com/paper/2303.02416/) (2023)
- [Improving anatomical plausibility in medical image segmentation via hybrid graph neural networks: applications to chest x-ray analysis](https://mlsanity.com/paper/2203.10977/) (2022)
- [Self-Supervised Pretraining for 2D Medical Image Segmentation](https://mlsanity.com/paper/2209.00314/) (2022)
- [Self-Supervised Pre-Training with Contrastive and Masked Autoencoder Methods for Dealing with Small Datasets in Deep Learning for Medical Imaging](https://mlsanity.com/paper/2308.06534/) (2023)
- [Weakly-Supervised Segmentation for Disease Localization in Chest X-Ray Images](https://mlsanity.com/paper/2007.00748/) (2020)
- [MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts](https://mlsanity.com/paper/2310.02000/) (2023)
- [Contrastive Cross-Modal Pre-Training: A General Strategy for Small Sample Medical Imaging](https://mlsanity.com/paper/2010.03060/) (2020)
- [Cross-Model Consensus of Explanations and Beyond for Image Classification Models: An Empirical Study](https://mlsanity.com/paper/2109.00707/) (2021)
- [Distilling Ensemble of Explanations for Weakly-Supervised Pre-Training of Image Segmentation Models](https://mlsanity.com/paper/2207.03335/) (2022)
- [Video4MRI: An Empirical Study on Brain Magnetic Resonance Image Analytics with CNN-based Video Classification Frameworks](https://mlsanity.com/paper/2302.12688/) (2023)

## Built on by (most cited first, 2 of 2)

- [Pre-trained Graphformer-based Ranking at Web-scale Search (Extended Abstract)](https://mlsanity.com/paper/2409.16590/) (2024)
- [Generative Pre-trained Ranking Model with Over-parameterization at Web-Scale (Extended Abstract)](https://mlsanity.com/paper/2409.16594/) (2024)

## Abstract (arXiv)

Medical image segmentation aims to identify and locate abnormal structures in medical images, such as chest radiographs, using deep neural networks. These networks require a large number of annotated images with fine-grained masks for the regions of interest, making pre-training strategies based on classification datasets essential for sample efficiency. Based on a large-scale medical image classification dataset, our work collects explanations from well-trained classifiers to generate pseudo labels of segmentation tasks. Specifically, we offer a case study on chest radiographs and train image classifiers on the CheXpert dataset to identify 14 pathological observations in radiology. We then use Integrated Gradients (IG) method to distill and boost the explanations obtained from the classifiers, generating massive diagnosis-oriented localization labels (DoLL). These DoLL-annotated images are used for pre-training the model before fine-tuning it for downstream segmentation tasks, including COVID-19 infectious areas, lungs, heart, and clavicles. Our method outperforms other baselines, showcasing significant advantages in model performance and training efficiency across various segmentation settings.