# Multi-task fusion for improving mammography screening data classification

arXiv: https://arxiv.org/abs/2112.01320 · HTML: https://mlsanity.com/paper/2112.01320/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 74 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 22 of 22)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [Speed/accuracy trade-offs for modern convolutional object detectors](https://mlsanity.com/paper/1611.10012/) (2016)
- [Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning](https://mlsanity.com/paper/1609.06570/) (2016)
- [Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening](https://mlsanity.com/paper/1903.08297/) (2019)
- [End-to-end Training for Whole Image Breast Cancer Diagnosis using An All Convolutional Design](https://mlsanity.com/paper/1711.05775/) (2017)
- [Detecting and classifying lesions in mammograms with Deep Learning](https://mlsanity.com/paper/1707.08401/) (2017)
- [An interpretable classifier for high-resolution breast cancer screening images utilizing weakly supervised localization](https://mlsanity.com/paper/2002.07613/) (2020)
- [Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification](https://mlsanity.com/paper/1612.05968/) (2016)
- [Robust breast cancer detection in mammography and digital breast tomosynthesis using annotation-efficient deep learning approach](https://mlsanity.com/paper/1912.11027/) (2019)
- [High-Resolution Breast Cancer Screening with Multi-View Deep Convolutional Neural Networks](https://mlsanity.com/paper/1703.07047/) (2017)
- [A Multi-Scale CNN and Curriculum Learning Strategy for Mammogram Classification](https://mlsanity.com/paper/1707.06978/) (2017)
- [Domain aware medical image classifier interpretation by counterfactual impact analysis](https://mlsanity.com/paper/2007.06312/) (2020)
- [Globally-Aware Multiple Instance Classifier for Breast Cancer Screening](https://mlsanity.com/paper/1906.02846/) (2019)
- [Interpretable Mammographic Image Classification using Case-Based Reasoning and Deep Learning](https://mlsanity.com/paper/2107.05605/) (2021)
- [Classifying Symmetrical Differences and Temporal Change in Mammography Using Deep Neural Networks](https://mlsanity.com/paper/1703.07715/) (2017)
- [Breast density classification with deep convolutional neural networks](https://mlsanity.com/paper/1711.03674/) (2017)
- [Interpreting Medical Image Classifiers by Optimization Based Counterfactual Impact Analysis](https://mlsanity.com/paper/2004.01610/) (2020)

## Built on by (most cited first, 4 of 4)

- [Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions](https://mlsanity.com/paper/2304.06662/) (2023)
- [Cross-Task Attention Network: Improving Multi-Task Learning for Medical Imaging Applications](https://mlsanity.com/paper/2309.03837/) (2023)
- [A Narrative Review on Large AI Models in Lung Cancer Screening, Diagnosis, and Treatment Planning](https://mlsanity.com/paper/2506.07236/) (2025)
- [CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report Generation](https://mlsanity.com/paper/2601.15408/) (2026)

## Abstract (arXiv)

Machine learning and deep learning methods have become essential for computer-assisted prediction in medicine, with a growing number of applications also in the field of mammography. Typically these algorithms are trained for a specific task, e.g., the classification of lesions or the prediction of a mammogram's pathology status. To obtain a comprehensive view of a patient, models which were all trained for the same task(s) are subsequently ensembled or combined. In this work, we propose a pipeline approach, where we first train a set of individual, task-specific models and subsequently investigate the fusion thereof, which is in contrast to the standard model ensembling strategy. We fuse model predictions and high-level features from deep learning models with hybrid patient models to build stronger predictors on patient level. To this end, we propose a multi-branch deep learning model which efficiently fuses features across different tasks and mammograms to obtain a comprehensive patient-level prediction. We train and evaluate our full pipeline on public mammography data, i.e., DDSM and its curated version CBIS-DDSM, and report an AUC score of 0.962 for predicting the presence of any lesion and 0.791 for predicting the presence of malignant lesions on patient level. Overall, our fusion approaches improve AUC scores significantly by up to 0.04 compared to standard model ensembling. Moreover, by providing not only global patient-level predictions but also task-specific model results that are related to radiological features, our pipeline aims to closely support the reading workflow of radiologists.