# Harnessing the power of longitudinal medical imaging for eye disease prognosis using Transformer-based sequence modeling

arXiv: https://arxiv.org/abs/2405.08780 · HTML: https://mlsanity.com/paper/2405.08780/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 54 references that are in the map.
- First posted on arXiv 14 May 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DeepSurv: Personalized Treatment Recommender System Using A Cox Proportional Hazards Deep Neural Network](https://mlsanity.com/paper/1606.00931/) (2016)
- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [Deep Learning for Survival Analysis: A Review](https://mlsanity.com/paper/2305.14961/) (2023)
- [COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction](https://mlsanity.com/paper/2101.04909/) (2021)
- [DeepSeeNet: A deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs](https://mlsanity.com/paper/1811.07492/) (2018)
- [Predicting risk of late age-related macular degeneration using deep learning](https://mlsanity.com/paper/2007.09550/) (2020)
- [A multi-task deep learning model for the classification of Age-related Macular Degeneration](https://mlsanity.com/paper/1812.00422/) (2018)
- [CNN-based Survival Model for Pancreatic Ductal Adenocarcinoma in Medical Imaging](https://mlsanity.com/paper/1906.10729/) (2019)
- [Predicting Time-to-conversion for Dementia of Alzheimer's Type using Multi-modal Deep Survival Analysis](https://mlsanity.com/paper/2205.01188/) (2022)
- [Deep survival analysis with longitudinal X-rays for COVID-19](https://mlsanity.com/paper/2108.09641/) (2021)

## Abstract (arXiv)

Deep learning has enabled breakthroughs in automated diagnosis from medical imaging, with many successful applications in ophthalmology. However, standard medical image classification approaches only assess disease presence at the time of acquisition, neglecting the common clinical setting of longitudinal imaging. For slow, progressive eye diseases like age-related macular degeneration (AMD) and primary open-angle glaucoma (POAG), patients undergo repeated imaging over time to track disease progression and forecasting the future risk of developing disease is critical to properly plan treatment. Our proposed Longitudinal Transformer for Survival Analysis (LTSA) enables dynamic disease prognosis from longitudinal medical imaging, modeling the time to disease from sequences of fundus photography images captured over long, irregular time periods. Using longitudinal imaging data from the Age-Related Eye Disease Study (AREDS) and Ocular Hypertension Treatment Study (OHTS), LTSA significantly outperformed a single-image baseline in 19/20 head-to-head comparisons on late AMD prognosis and 18/20 comparisons on POAG prognosis. A temporal attention analysis also suggested that, while the most recent image is typically the most influential, prior imaging still provides additional prognostic value.