# Multi-Modal MRI Reconstruction Assisted with Spatial Alignment Network arXiv: https://arxiv.org/abs/2108.05603 ยท HTML: https://mlsanity.com/paper/2108.05603/ ## Key facts - Cited by 16 papers in the map (data through 14 Oct 2026). - Builds on 21 of its 52 references that are in the map. - First posted on arXiv 12 Aug 2021. ## Builds on (most cited first, 21 of 21) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015) - [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016) - [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018) - [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018) - [A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1704.02422/) (2017) - [Image reconstruction by domain transform manifold learning](https://mlsanity.com/paper/1704.08841/) (2017) - [End-to-End Variational Networks for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2004.06688/) (2020) - [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018) - [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017) - [Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1712.01751/) (2017) - [DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Prior](https://mlsanity.com/paper/2001.03799/) (2020) - [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020) - [Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge](https://mlsanity.com/paper/2001.02518/) (2020) - [Multi-Contrast Super-Resolution MRI Through a Progressive Network](https://mlsanity.com/paper/1908.01612/) (2019) - [A Deep Information Sharing Network for Multi-contrast Compressed Sensing MRI Reconstruction](https://mlsanity.com/paper/1804.03596/) (2018) - [Coupled Dictionary Learning for Multi-contrast MRI Reconstruction](https://mlsanity.com/paper/1806.09930/) (2018) - [LAPNet: Non-rigid Registration derived in k-space for Magnetic Resonance Imaging](https://mlsanity.com/paper/2107.09060/) (2021) - [Compressed sensing for longitudinal MRI: An adaptive-weighted approach](https://mlsanity.com/paper/1407.2602/) (2014) ## Built on by (most cited first, 16 of 16) - [Adaptive Diffusion Priors for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2207.05876/) (2022) - [MMR-Mamba: Multi-Modal MRI Reconstruction with Mamba and Spatial-Frequency Information Fusion](https://mlsanity.com/paper/2406.18950/) (2024) - [On the Value of Cross-Modal Misalignment in Multimodal Representation Learning](https://mlsanity.com/paper/2504.10143/) (2025) - [Unsupervised Landmark Detection Based Spatiotemporal Motion Estimation for 4D Dynamic Medical Images](https://mlsanity.com/paper/2109.14805/) (2021) - [Spatial and Modal Optimal Transport for Fast Cross-Modal MRI Reconstruction](https://mlsanity.com/paper/2305.02774/) (2023) - [Federated Pseudo Modality Generation for Incomplete Multi-Modal MRI Reconstruction](https://mlsanity.com/paper/2308.10910/) (2023) - [JoJoNet: Joint-contrast and Joint-sampling-and-reconstruction Network for Multi-contrast MRI](https://mlsanity.com/paper/2210.12548/) (2022) - [A Plug-and-Play Method for Guided Multi-contrast MRI Reconstruction based on Content/Style Modeling](https://mlsanity.com/paper/2409.13477/) (2024) - [DuDoUniNeXt: Dual-domain unified hybrid model for single and multi-contrast undersampled MRI reconstruction](https://mlsanity.com/paper/2403.05256/) (2024) - [Re-Visible Dual-Domain Self-Supervised Deep Unfolding Network for MRI Reconstruction](https://mlsanity.com/paper/2501.03737/) (2025) - [A Comprehensive Survey on Magnetic Resonance Image Reconstruction](https://mlsanity.com/paper/2503.07097/) (2025) - [Bi-directional Self-Registration for Misaligned Infrared-Visible Image Fusion](https://mlsanity.com/paper/2505.06920/) (2025) - [Fusion for Visual-Infrared Person ReID in Real-World Surveillance Using Corrupted Multimodal Data](https://mlsanity.com/paper/2305.00320/) (2023) - [Decoupling Multi-Contrast Super-Resolution: Self-Supervised Implicit Re-Representation for Unpaired Cross-Modal Synthesis](https://mlsanity.com/paper/2505.05855/) (2025) - [Frequency Error-Guided Under-sampling Optimization for Multi-Contrast MRI Reconstruction](https://mlsanity.com/paper/2601.09316/) (2026) - [Robust Self-Supervised Cross-Modal Super-Resolution against Real-World Misaligned Observations](https://mlsanity.com/paper/2602.18822/) (2026) ## Abstract (arXiv) In clinical practice, multi-modal magnetic resonance imaging (MRI) with different contrasts is usually acquired in a single study to assess different properties of the same region of interest in the human body. The whole acquisition process can be accelerated by having one or more modalities under-sampled in the k-space. Recent research has shown that, considering the redundancy between different modalities, a target MRI modality under-sampled in the k-space can be more efficiently reconstructed with a fully-sampled reference MRI modality. However, we find that the performance of the aforementioned multi-modal reconstruction can be negatively affected by subtle spatial misalignment between different modalities, which is actually common in clinical practice. In this paper, we improve the quality of multi-modal reconstruction by compensating for such spatial misalignment with a spatial alignment network. First, our spatial alignment network estimates the displacement between the fully-sampled reference and the under-sampled target images, and warps the reference image accordingly. Then, the aligned fully-sampled reference image joins the multi-modal reconstruction of the under-sampled target image. Also, considering the contrast difference between the target and reference images, we have designed a cross-modality-synthesis-based registration loss in combination with the reconstruction loss, to jointly train the spatial alignment network and the reconstruction network. The experiments on both clinical MRI and multi-coil k-space raw data demonstrate the superiority and robustness of the multi-modal MRI reconstruction empowered with our spatial alignment network. Our code is publicly available at urlhttps://github.com/woxuankai/SpatialAlignmentNetwork.