# Attention-based Multi-scale Gated Recurrent Encoder with Novel Correlation Loss for COVID-19 Progression Prediction

arXiv: https://arxiv.org/abs/2107.08330 · HTML: https://mlsanity.com/paper/2107.08330/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 18 Jul 2021.

## Builds on (most cited first, 5 of 5)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Deep Multimodal Representation Learning from Temporal Data](https://mlsanity.com/paper/1704.03152/) (2017)
- [Convolutional Invasion and Expansion Networks for Tumor Growth Prediction](https://mlsanity.com/paper/1801.08468/) (2018)

## Built on by (most cited first, 5 of 5)

- [Test-time Adaptation with Calibration of Medical Image Classification Nets for Label Distribution Shift](https://mlsanity.com/paper/2207.00769/) (2022)
- [RadioTransformer: A Cascaded Global-Focal Transformer for Visual Attention-guided Disease Classification](https://mlsanity.com/paper/2202.11781/) (2022)
- [Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentation](https://mlsanity.com/paper/2302.04308/) (2023)
- [Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations](https://mlsanity.com/paper/2203.01933/) (2022)
- [Employing similarity to highlight differences: On the impact of anatomical assumptions in chest X-ray registration methods](https://mlsanity.com/paper/2301.09338/) (2023)

## Abstract (arXiv)

COVID-19 image analysis has mostly focused on diagnostic tasks using single timepoint scans acquired upon disease presentation or admission. We present a deep learning-based approach to predict lung infiltrate progression from serial chest radiographs (CXRs) of COVID-19 patients. Our method first utilizes convolutional neural networks (CNNs) for feature extraction from patches within the concerned lung zone, and also from neighboring and remote boundary regions. The framework further incorporates a multi-scale Gated Recurrent Unit (GRU) with a correlation module for effective predictions. The GRU accepts CNN feature vectors from three different areas as input and generates a fused representation. The correlation module attempts to minimize the correlation loss between hidden representations of concerned and neighboring area feature vectors, while maximizing the loss between the same from concerned and remote regions. Further, we employ an attention module over the output hidden states of each encoder timepoint to generate a context vector. This vector is used as an input to a decoder module to predict patch severity grades at a future timepoint. Finally, we ensemble the patch classification scores to calculate patient-wise grades. Specifically, our framework predicts zone-wise disease severity for a patient on a given day by learning representations from the previous temporal CXRs. Our novel multi-institutional dataset comprises sequential CXR scans from N=93 patients. Our approach outperforms transfer learning and radiomic feature-based baseline approaches on this dataset.