# Multimodal-Boost: Multimodal Medical Image Super-Resolution using Multi-Attention Network with Wavelet Transform

arXiv: https://arxiv.org/abs/2110.11684 · HTML: https://mlsanity.com/paper/2110.11684/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 55 references that are in the map.
- First posted on arXiv 22 Oct 2021.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [AMC: AutoML for Model Compression and Acceleration on Mobile Devices](https://mlsanity.com/paper/1802.03494/) (2018)
- [Deep Learning for Image Super-resolution: A Survey](https://mlsanity.com/paper/1902.06068/) (2019)
- [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018)
- [Deep Back-Projection Networks For Super-Resolution](https://mlsanity.com/paper/1803.02735/) (2018)
- [Meta-SR: A Magnification-Arbitrary Network for Super-Resolution](https://mlsanity.com/paper/1903.00875/) (2019)
- [CT Super-resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble(GAN-CIRCLE)](https://mlsanity.com/paper/1808.04256/) (2018)
- [Efficient and Accurate MRI Super-Resolution using a Generative Adversarial Network and 3D Multi-Level Densely Connected Network](https://mlsanity.com/paper/1803.01417/) (2018)
- [3D Convolutional Encoder-Decoder Network for Low-Dose CT via Transfer Learning from a 2D Trained Network](https://mlsanity.com/paper/1802.05656/) (2018)
- [DW-GAN: A Discrete Wavelet Transform GAN for NonHomogeneous Dehazing](https://mlsanity.com/paper/2104.08911/) (2021)
- [RAISR: Rapid and Accurate Image Super Resolution](https://mlsanity.com/paper/1606.01299/) (2016)
- [Face Super-Resolution Through Wasserstein GANs](https://mlsanity.com/paper/1705.02438/) (2017)
- [Perception Consistency Ultrasound Image Super-resolution via Self-supervised CycleGAN](https://mlsanity.com/paper/2012.14142/) (2020)
- [Lesion Focused Super-Resolution](https://mlsanity.com/paper/1810.06693/) (2018)
- [TWIST-GAN: Towards Wavelet Transform and Transferred GAN for Spatio-Temporal Single Image Super Resolution](https://mlsanity.com/paper/2104.10268/) (2021)

## Built on by (most cited first, 6 of 6)

- [SCPNet: Unsupervised Cross-modal Homography Estimation via Intra-modal Self-supervised Learning](https://mlsanity.com/paper/2407.08148/) (2024)
- [LSwinSR: UAV Imagery Super-Resolution based on Linear Swin Transformer](https://mlsanity.com/paper/2303.10232/) (2023)
- [DISGAN: Wavelet-informed Discriminator Guides GAN to MRI Super-resolution with Noise Cleaning](https://mlsanity.com/paper/2308.12084/) (2023)
- [SSHNet: Unsupervised Cross-modal Homography Estimation via Problem Reformulation and Split Optimization](https://mlsanity.com/paper/2409.17993/) (2024)
- [WaveHiT-SR: Hierarchical Wavelet Network for Efficient Image Super-Resolution](https://mlsanity.com/paper/2508.19927/) (2025)
- [Fine-tuned Generative Adversarial Network-based Model for Medical Image Super-Resolution](https://mlsanity.com/paper/2211.00577/) (2022)

## Abstract (arXiv)

Deep learning based single image super resolution (SISR) algorithms has revolutionized the overall diagnosis framework by continually improving the architectural components and training strategies associated with convolutional neural networks (CNN) on low-resolution images. However, existing work lacks in two ways: i) the SR output produced exhibits poor texture details, and often produce blurred edges, ii) most of the models have been developed for a single modality, hence, require modification to adapt to a new one. This work addresses (i) by proposing generative adversarial network (GAN) with deep multi-attention modules to learn high-frequency information from low-frequency data. Existing approaches based on the GAN have yielded good SR results; however, the texture details of their SR output have been experimentally confirmed to be deficient for medical images particularly. The integration of wavelet transform (WT) and GANs in our proposed SR model addresses the aforementioned limitation concerning textons. While the WT divides the LR image into multiple frequency bands, the transferred GAN uses multi-attention and upsample blocks to predict high-frequency components. Additionally, we present a learning method for training domain-specific classifiers as perceptual loss functions. Using a combination of multi-attention GAN loss and a perceptual loss function results in an efficient and reliable performance. Applying the same model for medical images from diverse modalities is challenging, our work addresses (ii) by training and performing on several modalities via transfer learning. Using two medical datasets, we validate our proposed SR network against existing state-of-the-art approaches and achieve promising results in terms of SSIM and PSNR.