# BrainRotViT: Transformer-ResNet Hybrid for Explainable Modeling of Brain Aging from 3D sMRI

arXiv: https://arxiv.org/abs/2511.15188 · HTML: https://mlsanity.com/paper/2511.15188/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 84 references that are in the map.
- First posted on arXiv 19 Nov 2025.

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Predicting brain age with deep learning from raw imaging data results in a reliable and heritable biomarker](https://mlsanity.com/paper/1612.02572/) (2016)
- [Global-Local Transformer for Brain Age Estimation](https://mlsanity.com/paper/2109.01663/) (2021)
- [Brain Age Estimation From MRI Using Cascade Networks with Ranking Loss](https://mlsanity.com/paper/2106.03052/) (2021)
- [Deep Learning for Brain Age Estimation: A Systematic Review](https://mlsanity.com/paper/2212.03868/) (2022)
- [Deep Relation Learning for Regression and Its Application to Brain Age Estimation](https://mlsanity.com/paper/2204.06598/) (2022)
- [Triamese-ViT: A 3D-Aware Method for Robust Brain Age Estimation from MRIs](https://mlsanity.com/paper/2401.09475/) (2024)
- [Estimating Brain Age with Global and Local Dependencies](https://mlsanity.com/paper/2209.08933/) (2022)
- [Brain Age Estimation with a Greedy Dual-Stream Model for Limited Datasets](https://mlsanity.com/paper/2407.04808/) (2024)

## Abstract (arXiv)

Accurate brain age estimation from structural MRI is a valuable biomarker for studying aging and neurodegeneration. Traditional regression and CNN-based methods face limitations such as manual feature engineering, limited receptive fields, and overfitting on heterogeneous data. Pure transformer models, while effective, require large datasets and high computational cost. We propose Brain ResNet over trained Vision Transformer (BrainRotViT), a hybrid architecture that combines the global context modeling of vision transformers (ViT) with the local refinement of residual CNNs. A ViT encoder is first trained on an auxiliary age and sex classification task to learn slice-level features. The frozen encoder is then applied to all sagittal slices to generate a 2D matrix of embedding vectors, which is fed into a residual CNN regressor that incorporates subject sex at the final fully-connected layer to estimate continuous brain age. Our method achieves an MAE of 3.34 years (Pearson r=0.98, Spearman ρ=0.97, R2=0.95) on validation across 11 MRI datasets encompassing more than 130 acquisition sites, outperforming baseline and state-of-the-art models. It also generalizes well across 4 independent cohorts with MAEs between 3.77 and 5.04 years. Analyses on the brain age gap (the difference between the predicted age and actual age) show that aging patterns are associated with Alzheimer's disease, cognitive impairment, and autism spectrum disorder. Model attention maps highlight aging-associated regions of the brain, notably the cerebellar vermis, precentral and postcentral gyri, temporal lobes, and medial superior frontal gyrus. Our results demonstrate that this method provides an efficient, interpretable, and generalizable framework for brain-age prediction, bridging the gap between CNN- and transformer-based approaches while opening new avenues for aging and neurodegeneration research.