# Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learning

arXiv: https://arxiv.org/abs/2109.03062 · HTML: https://mlsanity.com/paper/2109.03062/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 7 Sep 2021.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks](https://mlsanity.com/paper/2106.04489/) (2021)
- [Doctor AI: Predicting Clinical Events via Recurrent Neural Networks](https://mlsanity.com/paper/1511.05942/) (2015)
- [Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis](https://mlsanity.com/paper/1706.03446/) (2017)
- [Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration](https://mlsanity.com/paper/2102.04110/) (2021)
- [Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review](https://mlsanity.com/paper/2107.02975/) (2021)
- [Dilated Convolutional Attention Network for Medical Code Assignment from Clinical Text](https://mlsanity.com/paper/2009.14578/) (2020)
- [Phenotyping of Clinical Notes with Improved Document Classification Models Using Contextualized Neural Language Models](https://mlsanity.com/paper/1910.13664/) (2019)
- [MT-Clinical BERT: Scaling Clinical Information Extraction with Multitask Learning](https://mlsanity.com/paper/2004.10220/) (2020)
- [Improving Clinical Outcome Predictions Using Convolution over Medical Entities with Multimodal Learning](https://mlsanity.com/paper/2011.12349/) (2020)
- [Medical Code Assignment with Gated Convolution and Note-Code Interaction](https://mlsanity.com/paper/2010.06975/) (2020)
- [Multitask Recalibrated Aggregation Network for Medical Code Prediction](https://mlsanity.com/paper/2104.00952/) (2021)

## Built on by (most cited first, 2 of 2)

- [A Unified Review of Deep Learning for Automated Medical Coding](https://mlsanity.com/paper/2201.02797/) (2022)
- [Meta-Learning over Time for Destination Prediction Tasks](https://mlsanity.com/paper/2206.14801/) (2022)

## Abstract (arXiv)

Multitask deep learning has been applied to patient outcome prediction from text, taking clinical notes as input and training deep neural networks with a joint loss function of multiple tasks. However, the joint training scheme of multitask learning suffers from inter-task interference, and diagnosis prediction among the multiple tasks has the generalizability issue due to rare diseases or unseen diagnoses. To solve these challenges, we propose a hypernetwork-based approach that generates task-conditioned parameters and coefficients of multitask prediction heads to learn task-specific prediction and balance the multitask learning. We also incorporate semantic task information to improves the generalizability of our task-conditioned multitask model. Experiments on early and discharge notes extracted from the real-world MIMIC database show our method can achieve better performance on multitask patient outcome prediction than strong baselines in most cases. Besides, our method can effectively handle the scenario with limited information and improve zero-shot prediction on unseen diagnosis categories.