# TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction

arXiv: https://arxiv.org/abs/2301.10829 · HTML: https://mlsanity.com/paper/2301.10829/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 25 Jan 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation](https://mlsanity.com/paper/2107.00781/) (2021)
- [TransMed: Transformers Advance Multi-modal Medical Image Classification](https://mlsanity.com/paper/2103.05940/) (2021)
- [Is it Time to Replace CNNs with Transformers for Medical Images?](https://mlsanity.com/paper/2108.09038/) (2021)
- [Multi-Compound Transformer for Accurate Biomedical Image Segmentation](https://mlsanity.com/paper/2106.14385/) (2021)
- [COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction](https://mlsanity.com/paper/2101.04909/) (2021)
- [Global-Local Transformer for Brain Age Estimation](https://mlsanity.com/paper/2109.01663/) (2021)
- [Instance-based Vision Transformer for Subtyping of Papillary Renal Cell Carcinoma in Histopathological Image](https://mlsanity.com/paper/2106.12265/) (2021)
- [Vision Transformer using Low-level Chest X-ray Feature Corpus for COVID-19 Diagnosis and Severity Quantification](https://mlsanity.com/paper/2104.07235/) (2021)
- [Vision Transformer for Classification of Breast Ultrasound Images](https://mlsanity.com/paper/2110.14731/) (2021)
- [TransFusion: Multi-view Divergent Fusion for Medical Image Segmentation with Transformers](https://mlsanity.com/paper/2203.10726/) (2022)
- [RadioTransformer: A Cascaded Global-Focal Transformer for Visual Attention-guided Disease Classification](https://mlsanity.com/paper/2202.11781/) (2022)
- [Medical Transformer: Universal Brain Encoder for 3D MRI Analysis](https://mlsanity.com/paper/2104.13633/) (2021)
- [UNetFormer: A Unified Vision Transformer Model and Pre-Training Framework for 3D Medical Image Segmentation](https://mlsanity.com/paper/2204.00631/) (2022)
- [Indication as Prior Knowledge for Multimodal Disease Classification in Chest Radiographs with Transformers](https://mlsanity.com/paper/2202.06076/) (2022)
- [Prediction of Thrombectomy Functional Outcomes using Multimodal Data](https://mlsanity.com/paper/2005.13061/) (2020)
- [TranSiam: Fusing Multimodal Visual Features Using Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2204.12185/) (2022)

## Built on by (most cited first, 3 of 3)

- [Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatment](https://mlsanity.com/paper/2404.12634/) (2024)
- [Automatic Prediction of Stroke Treatment Outcomes: Latest Advances and Perspectives](https://mlsanity.com/paper/2412.04812/) (2024)
- [Outcome prediction and individualized treatment effect estimation in patients with large vessel occlusion stroke](https://mlsanity.com/paper/2507.03046/) (2025)

## Abstract (arXiv)

Acute ischaemic stroke, caused by an interruption in blood flow to brain tissue, is a leading cause of disability and mortality worldwide. The selection of patients for the most optimal ischaemic stroke treatment is a crucial step for a successful outcome, as the effect of treatment highly depends on the time to treatment. We propose a transformer-based multimodal network (TranSOP) for a classification approach that employs clinical metadata and imaging information, acquired on hospital admission, to predict the functional outcome of stroke treatment based on the modified Rankin Scale (mRS). This includes a fusion module to efficiently combine 3D non-contrast computed tomography (NCCT) features and clinical information. In comparative experiments using unimodal and multimodal data on the MRCLEAN dataset, we achieve a state-of-the-art AUC score of 0.85.