# Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatment

arXiv: https://arxiv.org/abs/2404.12634 · HTML: https://mlsanity.com/paper/2404.12634/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 28 references that are in the map.
- First posted on arXiv 19 Apr 2024.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [DeiT III: Revenge of the ViT](https://mlsanity.com/paper/2204.07118/) (2022)
- [Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentation](https://mlsanity.com/paper/2306.04811/) (2023)
- [A Multimodal Fusion Network For Student Emotion Recognition Based on Transformer and Tensor Product](https://mlsanity.com/paper/2403.08511/) (2024)
- [Utilizing the LightGBM Algorithm for Operator User Credit Assessment Research](https://mlsanity.com/paper/2403.14483/) (2024)
- [TranSOP: Transformer-based Multimodal Classification for Stroke Treatment Outcome Prediction](https://mlsanity.com/paper/2301.10829/) (2023)

## Built on by (most cited first, 11 of 11)

- [YOLO-PPA based Efficient Traffic Sign Detection for Cruise Control in Autonomous Driving](https://mlsanity.com/paper/2409.03320/) (2024)
- [AR Overlay: Training Image Pose Estimation on Curved Surface in a Synthetic Way](https://mlsanity.com/paper/2409.14577/) (2024)
- [A Neural Matrix Decomposition Recommender System Model based on the Multimodal Large Language Model](https://mlsanity.com/paper/2407.08942/) (2024)
- [Research on Key Technologies for Cross-Cloud Federated Training of Large Language Models](https://mlsanity.com/paper/2410.19130/) (2024)
- [Improved Unet model for brain tumor image segmentation based on ASPP-coordinate attention mechanism](https://mlsanity.com/paper/2409.08588/) (2024)
- [Enhancing Exchange Rate Forecasting with Explainable Deep Learning Models](https://mlsanity.com/paper/2410.19241/) (2024)
- [Analysis of Financial Risk Behavior Prediction Using Deep Learning and Big Data Algorithms](https://mlsanity.com/paper/2410.19394/) (2024)
- [Detecting and Classifying Defective Products in Images Using YOLO](https://mlsanity.com/paper/2412.16935/) (2024)
- [Multi-modal clothing recommendation model based on large model and VAE enhancement](https://mlsanity.com/paper/2410.02219/) (2024)
- [Comparing Baseline and Day-1 Diffusion MRI Using Multimodal Deep Embeddings for Stroke Outcome Prediction](https://mlsanity.com/paper/2512.02088/) (2025)
- [Vision-Core Guided Contrastive Learning for Balanced Multi-modal Prognosis Prediction of Stroke](https://mlsanity.com/paper/2605.14710/) (2026)

## Abstract (arXiv)

This study proposes a multi-modal fusion framework Multitrans based on the Transformer architecture and self-attention mechanism. This architecture combines the study of non-contrast computed tomography (NCCT) images and discharge diagnosis reports of patients undergoing stroke treatment, using a variety of methods based on Transformer architecture approach to predicting functional outcomes of stroke treatment. The results show that the performance of single-modal text classification is significantly better than single-modal image classification, but the effect of multi-modal combination is better than any single modality. Although the Transformer model only performs worse on imaging data, when combined with clinical meta-diagnostic information, both can learn better complementary information and make good contributions to accurately predicting stroke treatment effects..