# A Self-supervised Multimodal Deep Learning Approach to Differentiate Post-radiotherapy Progression from Pseudoprogression in Glioblastoma

arXiv: https://arxiv.org/abs/2502.03999 · HTML: https://mlsanity.com/paper/2502.03999/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 55 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Builds on (most cited first, 14 of 14)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [What Makes Training Multi-Modal Classification Networks Hard?](https://mlsanity.com/paper/1905.12681/) (2019)
- [Self-supervised learning methods and applications in medical imaging analysis: A survey](https://mlsanity.com/paper/2109.08685/) (2021)
- [Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review](https://mlsanity.com/paper/2301.03505/) (2023)
- [The University of California San Francisco Preoperative Diffuse Glioma MRI (UCSF-PDGM) Dataset](https://mlsanity.com/paper/2109.00356/) (2021)
- [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022)
- [Comprehensive Multimodal Deep Learning Survival Prediction Enabled by a Transformer Architecture: A Multicenter Study in Glioblastoma](https://mlsanity.com/paper/2405.12963/) (2024)
- [On self-supervised multi-modal representation learning: An application to Alzheimer's disease](https://mlsanity.com/paper/2012.13619/) (2020)
- [Deep Learning for Cancer Prognosis Prediction Using Portrait Photos by StyleGAN Embedding](https://mlsanity.com/paper/2306.14596/) (2023)

## Built on by (most cited first, 4 of 4)

- [Towards a deep learning approach for classifying treatment response in glioblastomas](https://mlsanity.com/paper/2504.18268/) (2025)
- [Benchmarking GPT-5 in Radiation Oncology: Measurable Gains, but Persistent Need for Expert Oversight](https://mlsanity.com/paper/2508.21777/) (2025)
- [Large-Scale Pre-training Enables Multimodal AI Differentiation of Radiation Necrosis from Brain Metastasis Progression on Routine MRI](https://mlsanity.com/paper/2511.18208/) (2025)
- [Timepoint-Specific Benchmarking of Deep Learning Models for Glioblastoma Follow-Up MRI](https://mlsanity.com/paper/2511.18595/) (2025)

## Abstract (arXiv)

Accurate differentiation of pseudoprogression (PsP) from True Progression (TP) following radiotherapy (RT) in glioblastoma (GBM) patients is crucial for optimal treatment planning. However, this task remains challenging due to the overlapping imaging characteristics of PsP and TP. This study therefore proposes a multimodal deep-learning approach utilizing complementary information from routine anatomical MR images, clinical parameters, and RT treatment planning information for improved predictive accuracy. The approach utilizes a self-supervised Vision Transformer (ViT) to encode multi-sequence MR brain volumes to effectively capture both global and local context from the high dimensional input. The encoder is trained in a self-supervised upstream task on unlabeled glioma MRI datasets from the open BraTS2021, UPenn-GBM, and UCSF-PDGM datasets to generate compact, clinically relevant representations from FLAIR and T1 post-contrast sequences. These encoded MR inputs are then integrated with clinical data and RT treatment planning information through guided cross-modal attention, improving progression classification accuracy. This work was developed using two datasets from different centers: the Burdenko Glioblastoma Progression Dataset (n = 59) for training and validation, and the GlioCMV progression dataset from the University Hospital Erlangen (UKER) (n = 20) for testing. The proposed method achieved an AUC of 75.3%, outperforming the current state-of-the-art data-driven approaches. Importantly, the proposed approach relies on readily available anatomical MRI sequences, clinical data, and RT treatment planning information, enhancing its clinical feasibility. The proposed approach addresses the challenge of limited data availability for PsP and TP differentiation and could allow for improved clinical decision-making and optimized treatment plans for GBM patients.