# Simultaneous Super-Resolution and Cross-Modality Synthesis of 3D Medical Images using Weakly-Supervised Joint Convolutional Sparse Coding

arXiv: https://arxiv.org/abs/1705.02596 · HTML: https://mlsanity.com/paper/1705.02596/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2017.

## Built on by (most cited first, 30 of 33)

- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [ResViT: Residual vision transformers for multi-modal medical image synthesis](https://mlsanity.com/paper/2106.16031/) (2021)
- [Self-Consistent Recursive Diffusion Bridge for Medical Image Translation](https://mlsanity.com/paper/2405.06789/) (2024)
- [I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling](https://mlsanity.com/paper/2405.14022/) (2024)
- [Multimodal Multi-Head Convolutional Attention with Various Kernel Sizes for Medical Image Super-Resolution](https://mlsanity.com/paper/2204.04218/) (2022)
- [2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution](https://mlsanity.com/paper/2406.06649/) (2024)
- [Exploring Separable Attention for Multi-Contrast MR Image Super-Resolution](https://mlsanity.com/paper/2109.01664/) (2021)
- [Fine-tuning deep learning model parameters for improved super-resolution of dynamic MRI with prior-knowledge](https://mlsanity.com/paper/2102.02711/) (2021)
- [Generative Adversarial Networks for Image Super-Resolution: A Survey](https://mlsanity.com/paper/2204.13620/) (2022)
- [SAM-DiffSR: Structure-Modulated Diffusion Model for Image Super-Resolution](https://mlsanity.com/paper/2402.17133/) (2024)
- [Downscaled Representation Matters: Improving Image Rescaling with Collaborative Downscaled Images](https://mlsanity.com/paper/2211.10643/) (2022)
- [Cross-Modality Neuroimage Synthesis: A Survey](https://mlsanity.com/paper/2202.06997/) (2022)
- [Multi-Attention Generative Adversarial Network for Remote Sensing Image Super-Resolution](https://mlsanity.com/paper/2107.06536/) (2021)
- [Uncovering the Over-smoothing Challenge in Image Super-Resolution: Entropy-based Quantification and Contrastive Optimization](https://mlsanity.com/paper/2201.01034/) (2022)
- [Uni-COAL: A Unified Framework for Cross-Modality Synthesis and Super-Resolution of MR Images](https://mlsanity.com/paper/2311.08225/) (2023)
- [Synthesizing MR Image Contrast Enhancement Using 3D High-resolution ConvNets](https://mlsanity.com/paper/2104.01592/) (2021)
- [SSL: A Self-similarity Loss for Improving Generative Image Super-resolution](https://mlsanity.com/paper/2408.05713/) (2024)
- [Scale-Aware Dynamic Network for Continuous-Scale Super-Resolution](https://mlsanity.com/paper/2110.15655/) (2021)
- [Optical Flow for Video Super-Resolution: A Survey](https://mlsanity.com/paper/2203.10462/) (2022)
- [CondiQuant: Condition Number Based Low-Bit Quantization for Image Super-Resolution](https://mlsanity.com/paper/2502.15478/) (2025)
- [Interactive Gadolinium-Free MRI Synthesis: A Transformer with Localization Prompt Learning](https://mlsanity.com/paper/2503.01265/) (2025)
- [SRSR: Enhancing Semantic Accuracy in Real-World Image Super-Resolution with Spatially Re-Focused Text-Conditioning](https://mlsanity.com/paper/2510.22534/) (2025)
- [Real-Time Super-Resolution for Real-World Images on Mobile Devices](https://mlsanity.com/paper/2206.01777/) (2022)
- [An Attentive-based Generative Model for Medical Image Synthesis](https://mlsanity.com/paper/2306.01562/) (2023)
- [Variable Augmented Network for Invertible Modality Synthesis-Fusion](https://mlsanity.com/paper/2109.00670/) (2021)
- [Super-Resolution Based Patch-Free 3D Image Segmentation with High-Frequency Guidance](https://mlsanity.com/paper/2210.14645/) (2022)
- [Universal Robustness via Median Randomized Smoothing for Real-World Super-Resolution](https://mlsanity.com/paper/2405.14934/) (2024)
- [Feasibility Assessment of Multitasking in MRI Neuroimaging Analysis: Tissue Segmentation, Cross-Modality Conversion and Bias correction](https://mlsanity.com/paper/2105.14986/) (2021)
- [Can Super Resolution be used to improve Human Pose Estimation in Low Resolution Scenarios?](https://mlsanity.com/paper/2107.02108/) (2021)
- [Deep learning techniques for blind image super-resolution: A high-scale multi-domain perspective evaluation](https://mlsanity.com/paper/2306.09426/) (2023)

## Abstract (arXiv)

Magnetic Resonance Imaging (MRI) offers high-resolution in vivo imaging and rich functional and anatomical multimodality tissue contrast. In practice, however, there are challenges associated with considerations of scanning costs, patient comfort, and scanning time that constrain how much data can be acquired in clinical or research studies. In this paper, we explore the possibility of generating high-resolution and multimodal images from low-resolution single-modality imagery. We propose the weakly-supervised joint convolutional sparse coding to simultaneously solve the problems of super-resolution (SR) and cross-modality image synthesis. The learning process requires only a few registered multimodal image pairs as the training set. Additionally, the quality of the joint dictionary learning can be improved using a larger set of unpaired images. To combine unpaired data from different image resolutions/modalities, a hetero-domain image alignment term is proposed. Local image neighborhoods are naturally preserved by operating on the whole image domain (as opposed to image patches) and using joint convolutional sparse coding. The paired images are enhanced in the joint learning process with unpaired data and an additional maximum mean discrepancy term, which minimizes the dissimilarity between their feature distributions. Experiments show that the proposed method outperforms state-of-the-art techniques on both SR reconstruction and simultaneous SR and cross-modality synthesis.