# Brain-ID: Learning Contrast-agnostic Anatomical Representations for Brain Imaging

arXiv: https://arxiv.org/abs/2311.16914 · HTML: https://mlsanity.com/paper/2311.16914/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 62 references that are in the map.
- First posted on arXiv 28 Nov 2023.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [SynthMorph: learning contrast-invariant registration without acquired images](https://mlsanity.com/paper/2004.10282/) (2020)
- [SynthStrip: Skull-Stripping for Any Brain Image](https://mlsanity.com/paper/2203.09974/) (2022)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [FastSurfer -- A fast and accurate deep learning based neuroimaging pipeline](https://mlsanity.com/paper/1910.03866/) (2019)
- [Foundational Models Defining a New Era in Vision: A Survey and Outlook](https://mlsanity.com/paper/2307.13721/) (2023)
- [Local Temperature Scaling for Probability Calibration](https://mlsanity.com/paper/2008.05105/) (2020)
- [SAM: Self-supervised Learning of Pixel-wise Anatomical Embeddings in Radiological Images](https://mlsanity.com/paper/2012.02383/) (2020)
- [Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets](https://mlsanity.com/paper/2209.02032/) (2022)
- [Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2105.07059/) (2021)
- [Joint super-resolution and synthesis of 1 mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrast](https://mlsanity.com/paper/2012.13340/) (2020)
- [Accurate Point Cloud Registration with Robust Optimal Transport](https://mlsanity.com/paper/2111.00648/) (2021)
- [A Contrast-Adaptive Method for Simultaneous Whole-Brain and Lesion Segmentation in Multiple Sclerosis](https://mlsanity.com/paper/2005.05135/) (2020)
- [SAMConvex: Fast Discrete Optimization for CT Registration using Self-supervised Anatomical Embedding and Correlation Pyramid](https://mlsanity.com/paper/2307.09727/) (2023)
- [Discovering Hidden Physics Behind Transport Dynamics](https://mlsanity.com/paper/2011.12222/) (2020)
- [Deep Decomposition for Stochastic Normal-Abnormal Transport](https://mlsanity.com/paper/2111.14777/) (2021)

## Built on by (most cited first, 11 of 11)

- [Conditional diffusion models for guided anomaly detection in brain images using fluid-driven anomaly randomization](https://mlsanity.com/paper/2506.10233/) (2025)
- [Unraveling Normal Anatomy via Fluid-Driven Anomaly Randomization](https://mlsanity.com/paper/2501.13370/) (2025)
- [Evaluating Synthetic Data Generation for Domain Generalization in Fetal Brain MRI Segmentation](https://mlsanity.com/paper/2411.06842/) (2024)
- [USB: Unified Synthetic Brain Framework for Bidirectional Pathology-Healthy Generation and Editing](https://mlsanity.com/paper/2512.00269/) (2025)
- [MS-Glance: Bio-Insipred Non-semantic Context Vectors and their Applications in Supervising Image Reconstruction](https://mlsanity.com/paper/2410.23577/) (2024)
- [Reference-Free 3D Reconstruction of Brain Dissection Slabs via Learned Atlas Coordinates](https://mlsanity.com/paper/2503.09963/) (2025)
- [BioVFM-21M: Benchmarking and Scaling Self-Supervised Vision Foundation Models for Biomedical Image Analysis](https://mlsanity.com/paper/2505.09329/) (2025)
- [Learning to Upscale 3D Segmentations in Neuroimaging](https://mlsanity.com/paper/2505.21697/) (2025)
- [Bridging Modalities: Joint Synthesis and Registration Framework for Aligning Diffusion MRI with T1-Weighted Images](https://mlsanity.com/paper/2601.11689/) (2026)
- [Adapting Frozen Mono-modal Backbones for Multi-modal Registration via Contrast-Agnostic Instance Optimization](https://mlsanity.com/paper/2603.26393/) (2026)
- [dMRI-F: A Foundation Model for Generalizable Diffusion MRI Analysis from Sparse Acquisitions](https://mlsanity.com/paper/2610.17968/) (2026)

## Abstract (arXiv)

Recent learning-based approaches have made astonishing advances in calibrated medical imaging like computerized tomography (CT), yet they struggle to generalize in uncalibrated modalities -- notably magnetic resonance (MR) imaging, where performance is highly sensitive to the differences in MR contrast, resolution, and orientation. This prevents broad applicability to diverse real-world clinical protocols. We introduce Brain-ID, an anatomical representation learning model for brain imaging. With the proposed "mild-to-severe" intra-subject generation, Brain-ID is robust to the subject-specific brain anatomy regardless of the appearance of acquired images (e.g., contrast, deformation, resolution, artifacts). Trained entirely on synthetic data, Brain-ID readily adapts to various downstream tasks through only one layer. We present new metrics to validate the intra- and inter-subject robustness of Brain-ID features, and evaluate their performance on four downstream applications, covering contrast-independent (anatomy reconstruction/contrast synthesis, brain segmentation), and contrast-dependent (super-resolution, bias field estimation) tasks. Extensive experiments on six public datasets demonstrate that Brain-ID achieves state-of-the-art performance in all tasks on different MRI modalities and CT, and more importantly, preserves its performance on low-resolution and small datasets. Code is available at https://github.com/peirong26/Brain-ID.