# Generalized Multi-Task Learning from Substantially Unlabeled Multi-Source Medical Image Data

arXiv: https://arxiv.org/abs/2110.13185 · HTML: https://mlsanity.com/paper/2110.13185/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 25 Oct 2021.

## Builds on (most cited first, 19 of 19)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [CE-Net: Context Encoder Network for 2D Medical Image Segmentation](https://mlsanity.com/paper/1903.02740/) (2019)
- [An Overview of Deep Semi-Supervised Learning](https://mlsanity.com/paper/2006.05278/) (2020)
- [Medical Image Analysis using Convolutional Neural Networks: A Review](https://mlsanity.com/paper/1709.02250/) (2017)
- [Joint Learning of Saliency Detection and Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1909.04161/) (2019)
- [Y-Net: Joint Segmentation and Classification for Diagnosis of Breast Biopsy Images](https://mlsanity.com/paper/1806.01313/) (2018)
- [A Feature Transfer Enabled Multi-Task Deep Learning Model on Medical Imaging](https://mlsanity.com/paper/1906.01828/) (2019)
- [Partly Supervised Multitask Learning](https://mlsanity.com/paper/2005.02523/) (2020)
- [Bridging Saliency Detection to Weakly Supervised Object Detection Based on Self-paced Curriculum Learning](https://mlsanity.com/paper/1703.01290/) (2017)
- [Joint segmentation and classification of retinal arteries/veins from fundus images](https://mlsanity.com/paper/1903.01330/) (2019)
- [Semi-Supervised Multi-Task Learning With Chest X-Ray Images](https://mlsanity.com/paper/1908.03693/) (2019)
- [MultiMix: Sparingly Supervised, Extreme Multitask Learning From Medical Images](https://mlsanity.com/paper/2010.14731/) (2020)
- [Progressive Adversarial Semantic Segmentation](https://mlsanity.com/paper/2005.04311/) (2020)

## Built on by (most cited first, 4 of 4)

- [Explainable COVID-19 Infections Identification and Delineation Using Calibrated Pseudo Labels](https://mlsanity.com/paper/2202.07422/) (2022)
- [Enhancing Self-Training Methods](https://mlsanity.com/paper/2301.07294/) (2023)
- [Post-COVID Highlights: Challenges and Solutions of AI Techniques for Swift Identification of COVID-19](https://mlsanity.com/paper/2311.06258/) (2023)
- [Multi-Task Learning for Integrated Automated Contouring and Voxel-Based Dose Prediction in Radiotherapy](https://mlsanity.com/paper/2411.18767/) (2024)

## Abstract (arXiv)

Deep learning-based models, when trained in a fully-supervised manner, can be effective in performing complex image analysis tasks, although contingent upon the availability of large labeled datasets. Especially in the medical imaging domain, however, expert image annotation is expensive, time-consuming, and prone to variability. Semi-supervised learning from limited quantities of labeled data has shown promise as an alternative. Maximizing knowledge gains from copious unlabeled data benefits semi-supervised learning models. Moreover, learning multiple tasks within the same model further improves its generalizability. We propose MultiMix, a new multi-task learning model that jointly learns disease classification and anatomical segmentation in a semi-supervised manner, while preserving explainability through a novel saliency bridge between the two tasks. Our experiments with varying quantities of multi-source labeled data in the training sets confirm the effectiveness of MultiMix in the simultaneous classification of pneumonia and segmentation of the lungs in chest X-ray images. Moreover, both in-domain and cross-domain evaluations across these tasks further showcase the potential of our model to adapt to challenging generalization scenarios.