# Joint multi-task learning improves weakly-supervised biomarker prediction in computational pathology

arXiv: https://arxiv.org/abs/2403.03891 · HTML: https://mlsanity.com/paper/2403.03891/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Conflict-Averse Gradient Descent for Multi-task Learning](https://mlsanity.com/paper/2110.14048/) (2021)
- [Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout](https://mlsanity.com/paper/2010.06808/) (2020)
- [Deep Learning-based Computational Pathology Predicts Origins for Cancers of Unknown Primary](https://mlsanity.com/paper/2006.13932/) (2020)
- [Auto-Lambda: Disentangling Dynamic Task Relationships](https://mlsanity.com/paper/2202.03091/) (2022)
- [One Model is All You Need: Multi-Task Learning Enables Simultaneous Histology Image Segmentation and Classification](https://mlsanity.com/paper/2203.00077/) (2022)
- [Regression-based Deep-Learning predicts molecular biomarkers from pathology slides](https://mlsanity.com/paper/2304.05153/) (2023)
- [Multi-task pre-training of deep neural networks for digital pathology](https://mlsanity.com/paper/2005.02561/) (2020)
- [Extending Unsupervised Neural Image Compression With Supervised Multitask Learning](https://mlsanity.com/paper/2004.07041/) (2020)
- [From Whole-slide Image to Biomarker Prediction: A Protocol for End-to-End Deep Learning in Computational Pathology](https://mlsanity.com/paper/2312.10944/) (2023)

## Built on by (most cited first, 2 of 2)

- [GCond: Gradient Conflict Resolution via Accumulation-based Stabilization for Large-Scale Multi-Task Learning](https://mlsanity.com/paper/2509.07252/) (2025)
- [Efficient and generalizable prediction of molecular alterations in multiple cancer cohorts using H&E whole slide images](https://mlsanity.com/paper/2407.15816/) (2024)

## Abstract (arXiv)

Deep Learning (DL) can predict biomarkers directly from digitized cancer histology in a weakly-supervised setting. Recently, the prediction of continuous biomarkers through regression-based DL has seen an increasing interest. Nonetheless, clinical decision making often requires a categorical outcome. Consequently, we developed a weakly-supervised joint multi-task Transformer architecture which has been trained and evaluated on four public patient cohorts for the prediction of two key predictive biomarkers, microsatellite instability (MSI) and homologous recombination deficiency (HRD), trained with auxiliary regression tasks related to the tumor microenvironment. Moreover, we perform a comprehensive benchmark of 16 approaches of task balancing for weakly-supervised joint multi-task learning in computational pathology. Using our novel approach, we improve over the state-of-the-art area under the receiver operating characteristic by +7.7% and +4.1%, as well as yielding better clustering of latent embeddings by +8% and +5% for the prediction of MSI and HRD in external cohorts, respectively.