# End-to-End Deep Learning for Structural Brain Imaging: A Unified Framework

arXiv: https://arxiv.org/abs/2502.18523 · HTML: https://mlsanity.com/paper/2502.18523/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 23 Feb 2025.

## Builds on (most cited first, 11 of 11)

- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [SynthStrip: Skull-Stripping for Any Brain Image](https://mlsanity.com/paper/2203.09974/) (2022)
- [FBNETGEN: Task-aware GNN-based fMRI Analysis via Functional Brain Network Generation](https://mlsanity.com/paper/2205.12465/) (2022)
- [DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation](https://mlsanity.com/paper/1904.08465/) (2019)
- [LT-Net: Label Transfer by Learning Reversible Voxel-wise Correspondence for One-shot Medical Image Segmentation](https://mlsanity.com/paper/2003.07072/) (2020)
- [Modeling the Probabilistic Distribution of Unlabeled Data forOne-shot Medical Image Segmentation](https://mlsanity.com/paper/2102.02033/) (2021)
- [A deep learning model for data-driven discovery of functional connectivity](https://mlsanity.com/paper/2112.04013/) (2021)
- [One-shot Joint Extraction, Registration and Segmentation of Neuroimaging Data](https://mlsanity.com/paper/2307.15198/) (2023)
- [Discriminative Feature Selection for Uncertain Graph Classification](https://mlsanity.com/paper/1301.6626/) (2013)

## Abstract (arXiv)

Brain imaging analysis is fundamental in neuroscience, providing valuable insights into brain structure and function. Traditional workflows follow a sequential pipeline-brain extraction, registration, segmentation, parcellation, network generation, and classification-treating each step as an independent task. These methods rely heavily on task-specific training data and expert intervention to correct intermediate errors, making them particularly burdensome for high-dimensional neuroimaging data, where annotations and quality control are costly and time-consuming. We introduce UniBrain, a unified end-to-end framework that integrates all processing steps into a single optimization process, allowing tasks to interact and refine each other. Unlike traditional approaches that require extensive task-specific annotations, UniBrain operates with minimal supervision, leveraging only low-cost labels (i.e., classification and extraction) and a single labeled atlas. By jointly optimizing extraction, registration, segmentation, parcellation, network generation, and classification, UniBrain enhances both accuracy and computational efficiency while significantly reducing annotation effort. Experimental results demonstrate its superiority over existing methods across multiple tasks, offering a more scalable and reliable solution for neuroimaging analysis. Our code and data can be found at https://github.com/Anonymous7852/UniBrain