# Task Transformer Network for Joint MRI Reconstruction and Super-Resolution

arXiv: https://arxiv.org/abs/2106.06742 · HTML: https://mlsanity.com/paper/2106.06742/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 12 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Image reconstruction by domain transform manifold learning](https://mlsanity.com/paper/1704.08841/) (2017)
- [Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1712.01751/) (2017)
- [Efficient and Accurate MRI Super-Resolution using a Generative Adversarial Network and 3D Multi-Level Densely Connected Network](https://mlsanity.com/paper/1803.01417/) (2018)
- [Multi-Contrast MRI Super-Resolution via a Multi-Stage Integration Network](https://mlsanity.com/paper/2105.08949/) (2021)
- [MRI Super-Resolution with Ensemble Learning and Complementary Priors](https://mlsanity.com/paper/1907.03063/) (2019)
- [MRI Reconstruction via Cascaded Channel-wise Attention Network](https://mlsanity.com/paper/1810.08229/) (2018)
- [Dual-Octave Convolution for Accelerated Parallel MR Image Reconstruction](https://mlsanity.com/paper/2104.05345/) (2021)
- [Fast Multi-class Dictionaries Learning with Geometrical Directions in MRI Reconstruction](https://mlsanity.com/paper/1503.02945/) (2015)
- [FCSR-GAN: Joint Face Completion and Super-resolution via Multi-task Learning](https://mlsanity.com/paper/1911.01045/) (2019)

## Built on by (most cited first, 30 of 58)

- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review](https://mlsanity.com/paper/2301.03505/) (2023)
- [Swin Transformer for Fast MRI](https://mlsanity.com/paper/2201.03230/) (2022)
- [ReconFormer: Accelerated MRI Reconstruction Using Recurrent Transformer](https://mlsanity.com/paper/2201.09376/) (2022)
- [Multi-Modal Transformer for Accelerated MR Imaging](https://mlsanity.com/paper/2106.14248/) (2021)
- [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022)
- [One Model to Synthesize Them All: Multi-contrast Multi-scale Transformer for Missing Data Imputation](https://mlsanity.com/paper/2204.13738/) (2022)
- [Learning Federated Visual Prompt in Null Space for MRI Reconstruction](https://mlsanity.com/paper/2303.16181/) (2023)
- [Specificity-Preserving Federated Learning for MR Image Reconstruction](https://mlsanity.com/paper/2112.05752/) (2021)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction](https://mlsanity.com/paper/2203.08213/) (2022)
- [Transformer-empowered Multi-scale Contextual Matching and Aggregation for Multi-contrast MRI Super-resolution](https://mlsanity.com/paper/2203.13963/) (2022)
- [DisC-Diff: Disentangled Conditional Diffusion Model for Multi-Contrast MRI Super-Resolution](https://mlsanity.com/paper/2303.13933/) (2023)
- [Surface Vision Transformers: Attention-Based Modelling applied to Cortical Analysis](https://mlsanity.com/paper/2203.16414/) (2022)
- [Cross-Modality High-Frequency Transformer for MR Image Super-Resolution](https://mlsanity.com/paper/2203.15314/) (2022)
- [A Systematic Survey of Deep Learning-based Single-Image Super-Resolution](https://mlsanity.com/paper/2109.14335/) (2021)
- [Multimodal Multi-Head Convolutional Attention with Various Kernel Sizes for Medical Image Super-Resolution](https://mlsanity.com/paper/2204.04218/) (2022)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [Exploring Separable Attention for Multi-Contrast MR Image Super-Resolution](https://mlsanity.com/paper/2109.01664/) (2021)
- [Enhancing Label-efficient Medical Image Segmentation with Text-guided Diffusion Models](https://mlsanity.com/paper/2407.05323/) (2024)
- [SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction](https://mlsanity.com/paper/2309.03335/) (2023)
- [Analysis of Deep Image Prior and Exploiting Self-Guidance for Image Reconstruction](https://mlsanity.com/paper/2402.04097/) (2024)
- [Cross-Modal Vertical Federated Learning for MRI Reconstruction](https://mlsanity.com/paper/2306.02673/) (2023)
- [DATR: Domain-adaptive transformer for multi-domain landmark detection](https://mlsanity.com/paper/2203.06433/) (2022)
- [Vision Transformers in Medical Imaging: A Review](https://mlsanity.com/paper/2211.10043/) (2022)
- [SuperFormer: Volumetric Transformer Architectures for MRI Super-Resolution](https://mlsanity.com/paper/2406.03359/) (2024)
- [Towards Instance-adaptive Inference for Federated Learning](https://mlsanity.com/paper/2308.06051/) (2023)
- [MRI Super-Resolution with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2511.16854/) (2025)
- [ViDSOD-100: A New Dataset and a Baseline Model for RGB-D Video Salient Object Detection](https://mlsanity.com/paper/2406.12536/) (2024)

## Abstract (arXiv)

The core problem of Magnetic Resonance Imaging (MRI) is the trade off between acceleration and image quality. Image reconstruction and super-resolution are two crucial techniques in Magnetic Resonance Imaging (MRI). Current methods are designed to perform these tasks separately, ignoring the correlations between them. In this work, we propose an end-to-end task transformer network (T2Net) for joint MRI reconstruction and super-resolution, which allows representations and feature transmission to be shared between multiple task to achieve higher-quality, super-resolved and motion-artifacts-free images from highly undersampled and degenerated MRI data. Our framework combines both reconstruction and super-resolution, divided into two sub-branches, whose features are expressed as queries and keys. Specifically, we encourage joint feature learning between the two tasks, thereby transferring accurate task information. We first use two separate CNN branches to extract task-specific features. Then, a task transformer module is designed to embed and synthesize the relevance between the two tasks. Experimental results show that our multi-task model significantly outperforms advanced sequential methods, both quantitatively and qualitatively.