# Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution

arXiv: https://arxiv.org/abs/2203.13963 · HTML: https://mlsanity.com/paper/2203.13963/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 48 references that are in the map.
- First posted on arXiv 26 Mar 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution](https://mlsanity.com/paper/2106.02299/) (2021)
- [DuDoRNet: Learning a Dual-Domain Recurrent Network for Fast MRI Reconstruction with Deep T1 Prior](https://mlsanity.com/paper/2001.03799/) (2020)
- [Task Transformer Network for Joint MRI Reconstruction and Super-Resolution](https://mlsanity.com/paper/2106.06742/) (2021)
- [Brain MRI Super Resolution Using 3D Deep Densely Connected Neural Networks](https://mlsanity.com/paper/1801.02728/) (2018)
- [Multi-Contrast Super-Resolution MRI Through a Progressive Network](https://mlsanity.com/paper/1908.01612/) (2019)
- [Multi-Contrast MRI Super-Resolution via a Multi-Stage Integration Network](https://mlsanity.com/paper/2105.08949/) (2021)
- [MRI Super-Resolution with Ensemble Learning and Complementary Priors](https://mlsanity.com/paper/1907.03063/) (2019)
- [Context-Sensitive Super-Resolution for Fast Fetal Magnetic Resonance Imaging](https://mlsanity.com/paper/1703.00035/) (2017)
- [High-Resolution Pelvic MRI Reconstruction Using a Generative Adversarial Network with Attention and Cyclic Loss](https://mlsanity.com/paper/2107.09989/) (2021)
- [Recon-GLGAN: A Global-Local context based Generative Adversarial Network for MRI Reconstruction](https://mlsanity.com/paper/1908.09262/) (2019)
- [Rapid Whole-Heart CMR with Single Volume Super-resolution](https://mlsanity.com/paper/1912.10503/) (2019)

## Built on by (most cited first, 30 of 30)

- [Single-subject Multi-contrast MRI Super-resolution via Implicit Neural Representations](https://mlsanity.com/paper/2303.15065/) (2023)
- [Rethinking Diffusion Model for Multi-Contrast MRI Super-Resolution](https://mlsanity.com/paper/2404.04785/) (2024)
- [SRFormerV2: Taking a Closer Look at Permuted Self-Attention for Image Super-Resolution](https://mlsanity.com/paper/2303.09735/) (2023)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [Cross-receptive Focused Inference Network for Lightweight Image Super-Resolution](https://mlsanity.com/paper/2207.02796/) (2022)
- [MMR-Mamba: Multi-Modal MRI Reconstruction with Mamba and Spatial-Frequency Information Fusion](https://mlsanity.com/paper/2406.18950/) (2024)
- [Hierarchical Neural Operator Transformer with Learnable Frequency-aware Loss Prior for Arbitrary-scale Super-resolution](https://mlsanity.com/paper/2405.12202/) (2024)
- [TransMRSR: Transformer-based Self-Distilled Generative Prior for Brain MRI Super-Resolution](https://mlsanity.com/paper/2306.06669/) (2023)
- [MRI Super-Resolution with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2511.16854/) (2025)
- [Deep Unfolding Convolutional Dictionary Model for Multi-Contrast MRI Super-resolution and Reconstruction](https://mlsanity.com/paper/2309.01171/) (2023)
- [Contourlet Refinement Gate Framework for Thermal Spectrum Distribution Regularized Infrared Image Super-Resolution](https://mlsanity.com/paper/2411.12530/) (2024)
- [DISGAN: Wavelet-informed Discriminator Guides GAN to MRI Super-resolution with Noise Cleaning](https://mlsanity.com/paper/2308.12084/) (2023)
- [ECLARE: Efficient cross-planar learning for anisotropic resolution enhancement](https://mlsanity.com/paper/2503.11787/) (2025)
- [Rethink Arbitrary Style Transfer with Transformer and Contrastive Learning](https://mlsanity.com/paper/2404.13584/) (2024)
- [MagicWorld: Towards Long-Horizon Stability for Interactive Video World Exploration](https://mlsanity.com/paper/2511.18886/) (2025)
- [MoEDiff-SR: Mixture of Experts-Guided Diffusion Model for Region-Adaptive MRI Super-Resolution](https://mlsanity.com/paper/2504.07308/) (2025)
- [Global and Local Mamba Network for Multi-Modality Medical Image Super-Resolution](https://mlsanity.com/paper/2504.10105/) (2025)
- [SSCM: A Spatial-Semantic Consistent Model for Multi-Contrast MRI Super-Resolution](https://mlsanity.com/paper/2509.18593/) (2025)
- [A residual dense vision transformer for medical image super-resolution with segmentation-based perceptual loss fine-tuning](https://mlsanity.com/paper/2302.11184/) (2023)
- [A Three-Player GAN for Super-Resolution in Magnetic Resonance Imaging](https://mlsanity.com/paper/2303.13900/) (2023)
- [Dual Arbitrary Scale Super-Resolution for Multi-Contrast MRI](https://mlsanity.com/paper/2307.02334/) (2023)
- [Enhance the Image: Super Resolution using Artificial Intelligence in MRI](https://mlsanity.com/paper/2406.13625/) (2024)
- [Compound Attention and Neighbor Matching Network for Multi-contrast MRI Super-resolution](https://mlsanity.com/paper/2307.02148/) (2023)
- [Unifying Dimensions: A Linear Adaptive Approach to Lightweight Image Super-Resolution](https://mlsanity.com/paper/2409.17597/) (2024)
- [Rapid Whole Brain Motion-robust Mesoscale In-vivo MR Imaging using Multi-scale Implicit Neural Representation](https://mlsanity.com/paper/2502.08634/) (2025)
- [Decoupling Multi-Contrast Super-Resolution: Self-Supervised Implicit Re-Representation for Unpaired Cross-Modal Synthesis](https://mlsanity.com/paper/2505.05855/) (2025)
- [CD-DPE: Dual-Prompt Expert Network Based on Convolutional Dictionary Feature Decoupling for Multi-Contrast MRI Super-Resolution](https://mlsanity.com/paper/2511.14014/) (2025)
- [Frequency Error-Guided Under-sampling Optimization for Multi-Contrast MRI Reconstruction](https://mlsanity.com/paper/2601.09316/) (2026)
- [Likelihood-Separable Diffusion Inference for Multi-Image MRI Super-Resolution](https://mlsanity.com/paper/2601.14030/) (2026)
- [Cardiac MRI Through-Plane Super-Resolution Guided by Reference and Memory](https://mlsanity.com/paper/2607.07581/) (2026)

## Abstract (arXiv)

Magnetic resonance imaging (MRI) can present multi-contrast images of the same anatomical structures, enabling multi-contrast super-resolution (SR) techniques. Compared with SR reconstruction using a single-contrast, multi-contrast SR reconstruction is promising to yield SR images with higher quality by leveraging diverse yet complementary information embedded in different imaging modalities. However, existing methods still have two shortcomings: (1) they neglect that the multi-contrast features at different scales contain different anatomical details and hence lack effective mechanisms to match and fuse these features for better reconstruction; and (2) they are still deficient in capturing long-range dependencies, which are essential for the regions with complicated anatomical structures. We propose a novel network to comprehensively address these problems by developing a set of innovative Transformer-empowered multi-scale contextual matching and aggregation techniques; we call it McMRSR. Firstly, we tame transformers to model long-range dependencies in both reference and target images. Then, a new multi-scale contextual matching method is proposed to capture corresponding contexts from reference features at different scales. Furthermore, we introduce a multi-scale aggregation mechanism to gradually and interactively aggregate multi-scale matched features for reconstructing the target SR MR image. Extensive experiments demonstrate that our network outperforms state-of-the-art approaches and has great potential to be applied in clinical practice. Codes are available at https://github.com/XAIMI-Lab/McMRSR.