# Multi-Dataset Multi-Task Learning for COVID-19 Prognosis

arXiv: https://arxiv.org/abs/2405.13771 · HTML: https://mlsanity.com/paper/2405.13771/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 11 of 11)

- [One Model To Learn Them All](https://mlsanity.com/paper/1706.05137/) (2017)
- [Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation and Diagnosis for COVID-19](https://mlsanity.com/paper/2004.02731/) (2020)
- [CheXtransfer: Performance and Parameter Efficiency of ImageNet Models for Chest X-Ray Interpretation](https://mlsanity.com/paper/2101.06871/) (2021)
- [Predicting COVID-19 Pneumonia Severity on Chest X-ray with Deep Learning](https://mlsanity.com/paper/2005.11856/) (2020)
- [Multimodal Explainability via Latent Shift applied to COVID-19 stratification](https://mlsanity.com/paper/2212.14084/) (2022)
- [Synergistic Learning of Lung Lobe Segmentation and Hierarchical Multi-Instance Classification for Automated Severity Assessment of COVID-19 in CT Images](https://mlsanity.com/paper/2005.03832/) (2020)
- [AIforCOVID: predicting the clinical outcomes in patients with COVID-19 applying AI to chest-X-rays. An Italian multicentre study](https://mlsanity.com/paper/2012.06531/) (2020)
- [Multi-objective optimization determines when, which and how to fuse deep networks: an application to predict COVID-19 outcomes](https://mlsanity.com/paper/2204.03772/) (2022)
- [COVID-MTL: Multitask Learning with Shift3D and Random-weighted Loss for Automated Diagnosis and Severity Assessment of COVID-19](https://mlsanity.com/paper/2012.05509/) (2020)
- [BIO-CXRNET: A Robust Multimodal Stacking Machine Learning Technique for Mortality Risk Prediction of COVID-19 Patients using Chest X-Ray Images and Clinical Data](https://mlsanity.com/paper/2206.07595/) (2022)
- [Distill-2MD-MTL: Data Distillation based on Multi-Dataset Multi-Domain Multi-Task Frame Work to Solve Face Related Tasksks, Multi Task Learning, Semi-Supervised Learning](https://mlsanity.com/paper/1907.03402/) (2019)

## Built on by (most cited first, 11 of 11)

- [Doctor-in-the-Loop: An Explainable, Multi-View Deep Learning Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer](https://mlsanity.com/paper/2502.17503/) (2025)
- [Class Balancing Diversity Multimodal Ensemble for Alzheimer's Disease Diagnosis and Early Detection](https://mlsanity.com/paper/2410.10374/) (2024)
- [Whole-Body Image-to-Image Translation for a Virtual Scanner in a Healthcare Digital Twin](https://mlsanity.com/paper/2503.15555/) (2025)
- [Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging](https://mlsanity.com/paper/2506.18434/) (2025)
- [Any-to-Any Vision-Language Model for Multimodal X-ray Imaging and Radiological Report Generation](https://mlsanity.com/paper/2505.01091/) (2025)
- [Lesion-Aware Generative Artificial Intelligence for Virtual Contrast-Enhanced Mammography in Breast Cancer](https://mlsanity.com/paper/2505.03018/) (2025)
- [Multimodal Doctor-in-the-Loop: A Clinically-Guided Explainable Framework for Predicting Pathological Response in Non-Small Cell Lung Cancer](https://mlsanity.com/paper/2505.01390/) (2025)
- [Beyond a Single Mode: GAN Ensembles for Diverse Medical Data Generation](https://mlsanity.com/paper/2503.24258/) (2025)
- [Cross-dataset Multivariate Time-series Model for Parkinson's Diagnosis via Keyboard Dynamics](https://mlsanity.com/paper/2510.15950/) (2025)
- [LUMEN: Longitudinal Multi-Modal Radiology Model for Prognosis and Diagnosis](https://mlsanity.com/paper/2602.21142/) (2026)
- [Evaluating Vision Language Model Adaptations for Radiology Report Generation in Low-Resource Languages](https://mlsanity.com/paper/2505.01096/) (2025)

## Abstract (arXiv)

In the fight against the COVID-19 pandemic, leveraging artificial intelligence to predict disease outcomes from chest radiographic images represents a significant scientific aim. The challenge, however, lies in the scarcity of large, labeled datasets with compatible tasks for training deep learning models without leading to overfitting. Addressing this issue, we introduce a novel multi-dataset multi-task training framework that predicts COVID-19 prognostic outcomes from chest X-rays (CXR) by integrating correlated datasets from disparate sources, distant from conventional multi-task learning approaches, which rely on datasets with multiple and correlated labeling schemes. Our framework hypothesizes that assessing severity scores enhances the model's ability to classify prognostic severity groups, thereby improving its robustness and predictive power. The proposed architecture comprises a deep convolutional network that receives inputs from two publicly available CXR datasets, AIforCOVID for severity prognostic prediction and BRIXIA for severity score assessment, and branches into task-specific fully connected output networks. Moreover, we propose a multi-task loss function, incorporating an indicator function, to exploit multi-dataset integration. The effectiveness and robustness of the proposed approach are demonstrated through significant performance improvements in prognosis classification tasks across 18 different convolutional neural network backbones in different evaluation strategies. This improvement is evident over single-task baselines and standard transfer learning strategies, supported by extensive statistical analysis, showing great application potential.