A Plug-and-Play Method for Guided Multi-contrast MRI Reconstruction based on Content/Style Modeling
Not on the map. Why?
Abstract (arXiv)
Since the various MR contrasts of a given anatomy contain redundant information, one contrast can be used to guide the reconstruction of another undersampled contrast acquired subsequently in the same session. To solve this reconstruction problem leveraging multi-contrast side information, several end-to-end learning-based methods have been proposed. However, a key challenge is the requirement for large paired training datasets comprising raw k-space data and aligned reference images. We propose a modular plug-and-play method, which requires no k-space training data and relies solely on partially paired image-domain datasets. A content/style model of two-contrast MR image data is first learned and subsequently applied as a plug-and-play operator in iterative reconstruction. The disentanglement of content and style allows explicit representation of contrast-independent and contrast-specific factors. Consequently, incorporating prior information into the reconstruction reduces to a simple replacement operation on the aliased content of the estimated image using high-quality content derived from the reference scan. Combining this operation with an MR data consistency step, followed by a corrective procedure for the content estimate, yields an iterative scheme. We name this novel approach PnP-CoSMo. It offers, by design, cross-contrast generalizability and provides an explanatory framework based on the shared and non-shared generative factors underlying the two given contrasts. We explore various aspects, including interpretability and convergence, via simulations. Furthermore, its practicality is demonstrated on the public NYU fastMRI DICOM dataset, showing equivalent or superior quality and greater generalizability compared to end-to-end methods. On two in-house multi-coil datasets, PnP-CoSMo enabled up to 32.6% greater acceleration over non-guided reconstruction at given SSIM.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 52 references that are in the map. First posted on arXiv 20 Sep 2024.
Builds on
27 of 52 references.
References · 27
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
- 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
- 2017Unsupervised Image-to-Image Translation Networkscited by 602
- 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 464
- 2017MoDL: Model Based Deep Learning Architecture for Inverse Problemscited by 384
- 2017A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstructioncited by 231
- 2018Image Synthesis in Multi-Contrast MRI with Conditional Generative Adversarial Networkscited by 131
- 2022Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imagingcited by 94
- 2019Plug and play methods for magnetic resonance imaging (long version)cited by 75
- 2020DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Priorcited by 72
- 2021Multi-Modal Transformer for Accelerated MR Imagingcited by 42
15 more
- 2021Diagonal Attention and Style-based GAN for Content-Style Disentanglement in Image Generation and Translationcited by 35
- 2019mustGAN: Multi-Stream Generative Adversarial Networks for MR Image Synthesiscited by 34
- 2020Unpaired Deep Learning for Accelerated MRI using Optimal Transport Driven CycleGANcited by 28
- 2024I2I-Mamba: Multi-modal medical image synthesis via selective state space modelingcited by 26
- 2020An Adaptive Intelligence Algorithm for Undersampled Knee MRI Reconstructioncited by 19
- 2023Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysiscited by 18
- 2018Coupled Dictionary Learning for Multi-contrast MRI Reconstructioncited by 16
- 2021Multi-Modal MRI Reconstruction Assisted with Spatial Alignment Networkcited by 16
- 2023MRI Reconstruction with Side Information using Diffusion Modelscited by 15
- 2015Multi-Contrast MRI Reconstruction with Structure-Guided Total Variationcited by 11
- 2021MR-Contrast-Aware Image-to-Image Translations with Generative Adversarial Networkscited by 10
- 2020The Surprising Effectiveness of Linear Unsupervised Image-to-Image Translationcited by 5
- 2023Spatial and Modal Optimal Transport for Fast Cross-Modal MRI Reconstructioncited by 5
- 2019Simultaneous use of Individual and Joint Regularization Terms in Compressive Sensing: Joint Reconstruction of Multi-Channel Multi-Contrast MRI Acquisitionscited by 2
- 2022Image Quality Assessment for Magnetic Resonance Imagingcited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.