# Longitudinal detection of radiological abnormalities with time-modulated LSTM

arXiv: https://arxiv.org/abs/1807.06144 · HTML: https://mlsanity.com/paper/1807.06144/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2018.

## Built on by (most cited first, 9 of 9)

- [Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing](https://mlsanity.com/paper/2301.04558/) (2023)
- [TransforMesh: A Transformer Network for Longitudinal modeling of Anatomical Meshes](https://mlsanity.com/paper/2109.00532/) (2021)
- [A hybrid CNN-RNN approach for survival analysis in a Lung Cancer Screening study](https://mlsanity.com/paper/2303.10789/) (2023)
- [Employing similarity to highlight differences: On the impact of anatomical assumptions in chest X-ray registration methods](https://mlsanity.com/paper/2301.09338/) (2023)
- [Multi-scale Spatio-temporal Transformer-based Imbalanced Longitudinal Learning for Glaucoma Forecasting from Irregular Time Series Images](https://mlsanity.com/paper/2402.13475/) (2024)
- [CARRNN: A Continuous Autoregressive Recurrent Neural Network for Deep Representation Learning from Sporadic Temporal Data](https://mlsanity.com/paper/2104.03739/) (2021)
- [CheXLearner: Text-Guided Fine-Grained Representation Learning for Progression Detection](https://mlsanity.com/paper/2505.06903/) (2025)
- [Shape Modeling of Longitudinal Medical Images: From Diffeomorphic Metric Mapping to Deep Learning](https://mlsanity.com/paper/2503.21489/) (2025)
- [Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction](https://mlsanity.com/paper/2607.04912/) (2026)

## Abstract (arXiv)

Convolutional neural networks (CNNs) have been successfully employed in recent years for the detection of radiological abnormalities in medical images such as plain x-rays. To date, most studies use CNNs on individual examinations in isolation and discard previously available clinical information. In this study we set out to explore whether Long-Short-Term-Memory networks (LSTMs) can be used to improve classification performance when modelling the entire sequence of radiographs that may be available for a given patient, including their reports. A limitation of traditional LSTMs, though, is that they implicitly assume equally-spaced observations, whereas the radiological exams are event-based, and therefore irregularly sampled. Using both a simulated dataset and a large-scale chest x-ray dataset, we demonstrate that a simple modification of the LSTM architecture, which explicitly takes into account the time lag between consecutive observations, can boost classification performance. Our empirical results demonstrate improved detection of commonly reported abnormalities on chest x-rays such as cardiomegaly, consolidation, pleural effusion and hiatus hernia.