# Transfer Learning for Clinical Time Series Analysis using Deep Neural Networks

arXiv: https://arxiv.org/abs/1904.00655 · HTML: https://mlsanity.com/paper/1904.00655/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2019.

## Built on by (most cited first, 22 of 22)

- [Self-Supervised Contrastive Pre-Training For Time Series via Time-Frequency Consistency](https://mlsanity.com/paper/2206.08496/) (2022)
- [Time Series Prediction using Deep Learning Methods in Healthcare](https://mlsanity.com/paper/2108.13461/) (2021)
- [Unsupervised pre-training of graph transformers on patient population graphs](https://mlsanity.com/paper/2207.10603/) (2022)
- [Cross-hospital Sepsis Early Detection via Semi-supervised Optimal Transport with Self-paced Ensemble](https://mlsanity.com/paper/2106.10352/) (2021)
- [A Systematic Review of NLP for Dementia -- Tasks, Datasets and Opportunities](https://mlsanity.com/paper/2409.19737/) (2024)
- [MELEP: A Novel Predictive Measure of Transferability in Multi-Label ECG Diagnosis](https://mlsanity.com/paper/2311.04224/) (2023)
- [AutoMap: Automatic Medical Code Mapping for Clinical Prediction Model Deployment](https://mlsanity.com/paper/2203.02446/) (2022)
- [Cross-center Early Sepsis Recognition by Medical Knowledge Guided Collaborative Learning for Data-scarce Hospitals](https://mlsanity.com/paper/2302.05702/) (2023)
- [Improving Prediction of Low-Prior Clinical Events with Simultaneous General Patient-State Representation Learning](https://mlsanity.com/paper/2106.14838/) (2021)
- [SilGAN: Generating driving maneuvers for scenario-based software-in-the-loop testing](https://mlsanity.com/paper/2107.07364/) (2021)
- [Transfer Learning Approaches for Knowledge Discovery in Grid-based Geo-Spatiotemporal Data](https://mlsanity.com/paper/2110.00841/) (2021)
- [A Systematic Evaluation of Generated Time Series and Their Effects in Self-Supervised Pretraining](https://mlsanity.com/paper/2408.07869/) (2024)
- [Accelerating Cerebral Diagnostics with BrainFusion: A Comprehensive MRI Tumor Framework](https://mlsanity.com/paper/2509.24149/) (2025)
- [Exploring Accurate and Transparent Domain Adaptation in Predictive Healthcare via Concept-Grounded Orthogonal Inference](https://mlsanity.com/paper/2602.12542/) (2026)
- [Neural forecasting at scale](https://mlsanity.com/paper/2109.09705/) (2021)
- [Performance of Deep Learning models with transfer learning for multiple-step-ahead forecasts in monthly time series](https://mlsanity.com/paper/2203.11196/) (2022)
- [A Comparison of Methods for Neural Network Aggregation](https://mlsanity.com/paper/2303.03488/) (2023)
- [Length of Stay prediction for Hospital Management using Domain Adaptation](https://mlsanity.com/paper/2306.16823/) (2023)
- [Modular Deep Learning for Multivariate Time-Series: Decoupling Imputation and Downstream Tasks](https://mlsanity.com/paper/2411.03941/) (2024)
- [HILGEN: Hierarchically-Informed Data Generation for Biomedical NER Using Knowledgebases and Large Language Models](https://mlsanity.com/paper/2503.04930/) (2025)
- [An explainable hierarchical self attention-based approach for tremor detection in the time domain](https://mlsanity.com/paper/2606.00461/) (2026)
- [Large Language Models are Approximate Survival Estimators](https://mlsanity.com/paper/2609.38181/) (2026)

## Abstract (arXiv)

Deep neural networks have shown promising results for various clinical prediction tasks. However, training deep networks such as those based on Recurrent Neural Networks (RNNs) requires large labeled data, significant hyper-parameter tuning effort and expertise, and high computational resources. In this work, we investigate as to what extent can transfer learning address these issues when using deep RNNs to model multivariate clinical time series. We consider two scenarios for transfer learning using RNNs: i) domain-adaptation, i.e., leveraging a deep RNN - namely, TimeNet - pre-trained for feature extraction on time series from diverse domains, and adapting it for feature extraction and subsequent target tasks in healthcare domain, ii) task-adaptation, i.e., pre-training a deep RNN - namely, HealthNet - on diverse tasks in healthcare domain, and adapting it to new target tasks in the same domain. We evaluate the above approaches on publicly available MIMIC-III benchmark dataset, and demonstrate that (a) computationally-efficient linear models trained using features extracted via pre-trained RNNs outperform or, in the worst case, perform as well as deep RNNs and statistical hand-crafted features based models trained specifically for target task; (b) models obtained by adapting pre-trained models for target tasks are significantly more robust to the size of labeled data compared to task-specific RNNs, while also being computationally efficient. We, therefore, conclude that pre-trained deep models like TimeNet and HealthNet allow leveraging the advantages of deep learning for clinical time series analysis tasks, while also minimize dependence on hand-crafted features, deal robustly with scarce labeled training data scenarios without overfitting, as well as reduce dependence on expertise and resources required to train deep networks from scratch.