# A Plug-and-Play Method for Guided Multi-contrast MRI Reconstruction based on Content/Style Modeling

arXiv: https://arxiv.org/abs/2409.13477 · HTML: https://mlsanity.com/paper/2409.13477/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 52 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 27 of 27)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [MoDL: Model Based Deep Learning Architecture for Inverse Problems](https://mlsanity.com/paper/1712.02862/) (2017)
- [A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1704.02422/) (2017)
- [Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1802.01221/) (2018)
- [Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging](https://mlsanity.com/paper/2203.17061/) (2022)
- [Plug and play methods for magnetic resonance imaging (long version)](https://mlsanity.com/paper/1903.08616/) (2019)
- [DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Prior](https://mlsanity.com/paper/2001.03799/) (2020)
- [Multi-Modal Transformer for Accelerated MR Imaging](https://mlsanity.com/paper/2106.14248/) (2021)
- [Diagonal Attention and Style-based GAN for Content-Style Disentanglement in Image Generation and Translation](https://mlsanity.com/paper/2103.16146/) (2021)
- [mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesis](https://mlsanity.com/paper/1909.11504/) (2019)
- [Unpaired Deep Learning for Accelerated MRI using Optimal Transport Driven CycleGAN](https://mlsanity.com/paper/2008.12967/) (2020)
- [I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling](https://mlsanity.com/paper/2405.14022/) (2024)
- [An Adaptive Intelligence Algorithm for Undersampled Knee MRI Reconstruction](https://mlsanity.com/paper/2004.07339/) (2020)
- [Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysis](https://mlsanity.com/paper/2304.10278/) (2023)
- [Coupled Dictionary Learning for Multi-contrast MRI Reconstruction](https://mlsanity.com/paper/1806.09930/) (2018)
- [Multi-Modal MRI Reconstruction Assisted with Spatial Alignment Network](https://mlsanity.com/paper/2108.05603/) (2021)
- [MRI Reconstruction with Side Information using Diffusion Models](https://mlsanity.com/paper/2303.14795/) (2023)
- [Multi-Contrast MRI Reconstruction with Structure-Guided Total Variation](https://mlsanity.com/paper/1511.06631/) (2015)
- [MR-Contrast-Aware Image-to-Image Translations with Generative Adversarial Networks](https://mlsanity.com/paper/2104.01449/) (2021)
- [The Surprising Effectiveness of Linear Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2007.12568/) (2020)
- [Spatial and Modal Optimal Transport for Fast Cross-Modal MRI Reconstruction](https://mlsanity.com/paper/2305.02774/) (2023)
- [Simultaneous use of Individual and Joint Regularization Terms in Compressive Sensing: Joint Reconstruction of Multi-Channel Multi-Contrast MRI Acquisitions](https://mlsanity.com/paper/1903.00691/) (2019)
- [Image Quality Assessment for Magnetic Resonance Imaging](https://mlsanity.com/paper/2203.07809/) (2022)

## Built on by (most cited first, 3 of 3)

- [Generative Autoregressive Transformers for Model-Agnostic Federated MRI Reconstruction](https://mlsanity.com/paper/2502.04521/) (2025)
- [Data-Efficient Networks for Multi-Contrast MRI Reconstruction based on a Generalized Content/Style Prior](https://mlsanity.com/paper/2609.01959/) (2026)
- [Learning Subject-Specific Anatomical Representations via Manifold Expansion: Application to Accelerated Multi-Contrast MRI](https://mlsanity.com/paper/2610.04028/) (2026)

## Abstract (arXiv)

Since the various MR contrasts of a given anatomy contain redundant information, one contrast can be used to guide the reconstruction of another undersampled contrast acquired subsequently in the same session. To solve this reconstruction problem leveraging multi-contrast side information, several end-to-end learning-based methods have been proposed. However, a key challenge is the requirement for large paired training datasets comprising raw k-space data and aligned reference images. We propose a modular plug-and-play method, which requires no k-space training data and relies solely on partially paired image-domain datasets. A content/style model of two-contrast MR image data is first learned and subsequently applied as a plug-and-play operator in iterative reconstruction. The disentanglement of content and style allows explicit representation of contrast-independent and contrast-specific factors. Consequently, incorporating prior information into the reconstruction reduces to a simple replacement operation on the aliased content of the estimated image using high-quality content derived from the reference scan. Combining this operation with an MR data consistency step, followed by a corrective procedure for the content estimate, yields an iterative scheme. We name this novel approach PnP-CoSMo. It offers, by design, cross-contrast generalizability and provides an explanatory framework based on the shared and non-shared generative factors underlying the two given contrasts. We explore various aspects, including interpretability and convergence, via simulations. Furthermore, its practicality is demonstrated on the public NYU fastMRI DICOM dataset, showing equivalent or superior quality and greater generalizability compared to end-to-end methods. On two in-house multi-coil datasets, PnP-CoSMo enabled up to 32.6% greater acceleration over non-guided reconstruction at given SSIM.