# PEPSI: Pathology-Enhanced Pulse-Sequence-Invariant Representations for Brain MRI

arXiv: https://arxiv.org/abs/2403.06227 · HTML: https://mlsanity.com/paper/2403.06227/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 10 Mar 2024.

## Builds on (most cited first, 19 of 19)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [SynthMorph: learning contrast-invariant registration without acquired images](https://mlsanity.com/paper/2004.10282/) (2020)
- [SynthStrip: Skull-Stripping for Any Brain Image](https://mlsanity.com/paper/2203.09974/) (2022)
- [Foundational Models Defining a New Era in Vision: A Survey and Outlook](https://mlsanity.com/paper/2307.13721/) (2023)
- [ISLES 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset](https://mlsanity.com/paper/2206.06694/) (2022)
- [Joint super-resolution and synthesis of 1 mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrast](https://mlsanity.com/paper/2012.13340/) (2020)
- [Quantifying white matter hyperintensity and brain volumes in heterogeneous clinical and low-field portable MRI](https://mlsanity.com/paper/2312.05119/) (2023)
- [Discovering Hidden Physics Behind Transport Dynamics](https://mlsanity.com/paper/2011.12222/) (2020)
- [Deep Decomposition for Stochastic Normal-Abnormal Transport](https://mlsanity.com/paper/2111.14777/) (2021)
- [Differential Motion Evolution for Fine-Grained Motion Deformation in Unsupervised Image Animation](https://mlsanity.com/paper/2110.04658/) (2021)

## Built on by (most cited first, 8 of 8)

- [Learning General-Purpose Biomedical Volume Representations using Randomized Synthesis](https://mlsanity.com/paper/2411.02372/) (2024)
- [Conditional diffusion models for guided anomaly detection in brain images using fluid-driven anomaly randomization](https://mlsanity.com/paper/2506.10233/) (2025)
- [Advances in Automated Fetal Brain MRI Segmentation and Biometry: Insights from the FeTA 2024 Challenge](https://mlsanity.com/paper/2505.02784/) (2025)
- [Unraveling Normal Anatomy via Fluid-Driven Anomaly Randomization](https://mlsanity.com/paper/2501.13370/) (2025)
- [Evaluating Synthetic Data Generation for Domain Generalization in Fetal Brain MRI Segmentation](https://mlsanity.com/paper/2411.06842/) (2024)
- [Unified 3D MRI Representations via Sequence-Invariant Contrastive Learning](https://mlsanity.com/paper/2501.12057/) (2025)
- [USB: Unified Synthetic Brain Framework for Bidirectional Pathology-Healthy Generation and Editing](https://mlsanity.com/paper/2512.00269/) (2025)
- [MS-Glance: Bio-Insipred Non-semantic Context Vectors and their Applications in Supervising Image Reconstruction](https://mlsanity.com/paper/2410.23577/) (2024)

## Abstract (arXiv)

Remarkable progress has been made by data-driven machine-learning methods in the analysis of MRI scans. However, most existing MRI analysis approaches are crafted for specific MR pulse sequences (MR contrasts) and usually require nearly isotropic acquisitions. This limits their applicability to diverse real-world clinical data, where scans commonly exhibit variations in appearances due to being obtained with varying sequence parameters, resolutions, and orientations -- especially in the presence of pathology. In this paper, we propose PEPSI, the first pathology-enhanced, and pulse-sequence-invariant feature representation learning model for brain MRI. PEPSI is trained entirely on synthetic images with a novel pathology encoding strategy, and enables co-training across datasets with diverse pathologies and missing modalities. Despite variations in pathology appearances across different MR pulse sequences or the quality of acquired images (e.g., resolution, orientation, artifacts, etc), PEPSI produces a high-resolution image of reference contrast (MP-RAGE) that captures anatomy, along with an image specifically highlighting the pathology. Our experiments demonstrate PEPSI's remarkable capability for image synthesis compared with the state-of-the-art, contrast-agnostic synthesis models, as it accurately reconstructs anatomical structures while differentiating between pathology and normal tissue. We further illustrate the efficiency and effectiveness of PEPSI features for downstream pathology segmentations on five public datasets covering white matter hyperintensities and stroke lesions. Code is available at https://github.com/peirong26/PEPSI.