# Deep multi-modal aggregation network for MR image reconstruction with auxiliary modality

arXiv: https://arxiv.org/abs/2110.08080 · HTML: https://mlsanity.com/paper/2110.08080/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [MoDL: Model Based Deep Learning Architecture for Inverse Problems](https://mlsanity.com/paper/1712.02862/) (2017)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Multimodal Intelligence: Representation Learning, Information Fusion, and Applications](https://mlsanity.com/paper/1911.03977/) (2019)
- [Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss](https://mlsanity.com/paper/1709.00753/) (2017)
- [Recalibrating Fully Convolutional Networks with Spatial and Channel 'Squeeze & Excitation' Blocks](https://mlsanity.com/paper/1808.08127/) (2018)
- [DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Prior](https://mlsanity.com/paper/2001.03799/) (2020)
- [GrappaNet: Combining Parallel Imaging with Deep Learning for Multi-Coil MRI Reconstruction](https://mlsanity.com/paper/1910.12325/) (2019)
- [Learning MRI Artifact Removal With Unpaired Data](https://mlsanity.com/paper/2110.04604/) (2021)
- [Deep De-Aliasing for Fast Compressive Sensing MRI](https://mlsanity.com/paper/1705.07137/) (2017)

## Built on by (most cited first, 2 of 2)

- [Specificity-Preserving Federated Learning for MR Image Reconstruction](https://mlsanity.com/paper/2112.05752/) (2021)
- [A Comprehensive Survey on Magnetic Resonance Image Reconstruction](https://mlsanity.com/paper/2503.07097/) (2025)

## Abstract (arXiv)

Magnetic resonance (MR) imaging produces detailed images of organs and tissues with better contrast, but it suffers from a long acquisition time, which makes the image quality vulnerable to say motion artifacts. Recently, many approaches have been developed to reconstruct full-sampled images from partially observed measurements to accelerate MR imaging. However, most approaches focused on reconstruction over a single modality, neglecting the discovery of correlation knowledge between the different modalities. Here we propose a Multi-modal Aggregation network for mR Image recOnstruction with auxiliary modality (MARIO), which is capable of discovering complementary representations from a fully sampled auxiliary modality, with which to hierarchically guide the reconstruction of a given target modality. This implies that our method can selectively aggregate multi-modal representations for better reconstruction, yielding comprehensive, multi-scale, multi-modal feature fusion. Extensive experiments on IXI and fastMRI datasets demonstrate the superiority of the proposed approach over state-of-the-art MR image reconstruction methods in removing artifacts.