# PT: A Plain Transformer is Good Hospital Readmission Predictor

arXiv: https://arxiv.org/abs/2412.12909 · HTML: https://mlsanity.com/paper/2412.12909/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 54 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs](https://mlsanity.com/paper/1901.07042/) (2019)
- [ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission](https://mlsanity.com/paper/1904.05342/) (2019)
- [Multi-modal Transformer for Video Retrieval](https://mlsanity.com/paper/2007.10639/) (2020)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [Attention Bottlenecks for Multimodal Fusion](https://mlsanity.com/paper/2107.00135/) (2021)
- [Medical Transformer: Gated Axial-Attention for Medical Image Segmentation](https://mlsanity.com/paper/2102.10662/) (2021)
- [MM-ViT: Multi-Modal Video Transformer for Compressed Video Action Recognition](https://mlsanity.com/paper/2108.09322/) (2021)
- [Natural Language Processing of Clinical Notes on Chronic Diseases: Systematic Review](https://mlsanity.com/paper/1908.05780/) (2019)
- [Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies](https://mlsanity.com/paper/2107.09951/) (2021)
- [Predicting 30-Day Hospital Readmission in Medicare Patients: Insights from an LSTM Deep Learning Model](https://mlsanity.com/paper/2410.17545/) (2024)
- [Predicting Heart Failure Readmission from Clinical Notes Using Deep Learning](https://mlsanity.com/paper/1912.10306/) (2019)
- [Clinical Assistant Diagnosis for Electronic Medical Record Based on Convolutional Neural Network](https://mlsanity.com/paper/1804.08261/) (2018)
- [Stabilizing Machine Learning Prediction of Dynamics: Noise and Noise-inspired Regularization](https://mlsanity.com/paper/2211.05262/) (2022)
- [MuST: Multimodal Spatiotemporal Graph-Transformer for Hospital Readmission Prediction](https://mlsanity.com/paper/2311.07608/) (2023)

## Abstract (arXiv)

Hospital readmission prediction is critical for clinical decision support, aiming to identify patients at risk of returning within 30 days post-discharge. High readmission rates often indicate inadequate treatment or post-discharge care, making effective prediction models essential for optimizing resources and improving patient outcomes. We propose PT, a Transformer-based model that integrates Electronic Health Records (EHR), medical images, and clinical notes to predict 30-day all-cause hospital readmissions. PT extracts features from raw data and uses specialized Transformer blocks tailored to the data's complexity. Enhanced with Random Forest for EHR feature selection and test-time ensemble techniques, PT achieves superior accuracy, scalability, and robustness. It performs well even when temporal information is missing. Our main contributions are: (1)Simplicity: A powerful and efficient baseline model outperforming existing ones in prediction accuracy; (2)Scalability: Flexible handling of various features from different modalities, achieving high performance with just clinical notes or EHR data; (3)Robustness: Strong predictive performance even with missing or unclear temporal data.