# Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers

arXiv: https://arxiv.org/abs/2105.08059 · HTML: https://mlsanity.com/paper/2105.08059/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 111 references that are in the map.
- First posted on arXiv 15 May 2021.

## Builds on (most cited first, 30 of 58)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [MoDL: Model Based Deep Learning Architecture for Inverse Problems](https://mlsanity.com/paper/1712.02862/) (2017)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [Image reconstruction by domain transform manifold learning](https://mlsanity.com/paper/1704.08841/) (2017)
- [End-to-End Variational Networks for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2004.06688/) (2020)
- [Generative Adversarial Transformers](https://mlsanity.com/paper/2103.01209/) (2021)
- [Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1712.01751/) (2017)
- [Invertible generative models for inverse problems: mitigating representation error and dataset bias](https://mlsanity.com/paper/1905.11672/) (2019)
- [Deep learning for undersampled MRI reconstruction](https://mlsanity.com/paper/1709.02576/) (2017)
- [A Deep Cascade of Convolutional Neural Networks for MR Image Reconstruction](https://mlsanity.com/paper/1703.00555/) (2017)
- [Time-Dependent Deep Image Prior for Dynamic MRI](https://mlsanity.com/paper/1910.01684/) (2019)
- [RARE: Image Reconstruction using Deep Priors Learned without Ground Truth](https://mlsanity.com/paper/1912.05854/) (2019)
- [Deep Residual Learning for Accelerated MRI using Magnitude and Phase Networks](https://mlsanity.com/paper/1804.00432/) (2018)
- [A Transfer-Learning Approach for Accelerated MRI using Deep Neural Networks](https://mlsanity.com/paper/1710.02615/) (2017)
- [Compressive sensing with un-trained neural networks: Gradient descent finds the smoothest approximation](https://mlsanity.com/paper/2005.03991/) (2020)

## Built on by (most cited first, 30 of 77)

- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [ResViT: Residual vision transformers for multi-modal medical image synthesis](https://mlsanity.com/paper/2106.16031/) (2021)
- [Adaptive Diffusion Priors for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2207.05876/) (2022)
- [Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Review](https://mlsanity.com/paper/2301.03505/) (2023)
- [Swin Transformer for Fast MRI](https://mlsanity.com/paper/2201.03230/) (2022)
- [ReconFormer: Accelerated MRI Reconstruction Using Recurrent Transformer](https://mlsanity.com/paper/2201.09376/) (2022)
- [Multi-Modal Transformer for Accelerated MR Imaging](https://mlsanity.com/paper/2106.14248/) (2021)
- [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022)
- [One Model to Synthesize Them All: Multi-contrast Multi-scale Transformer for Missing Data Imputation](https://mlsanity.com/paper/2204.13738/) (2022)
- [Equitable modelling of brain imaging by counterfactual augmentation with morphologically constrained 3D deep generative models](https://mlsanity.com/paper/2111.14923/) (2021)
- [Specificity-Preserving Federated Learning for MR Image Reconstruction](https://mlsanity.com/paper/2112.05752/) (2021)
- [Class-Aware Adversarial Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2201.10737/) (2022)
- [Federated Learning of Generative Image Priors for MRI Reconstruction](https://mlsanity.com/paper/2202.04175/) (2022)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [DSFormer: A Dual-domain Self-supervised Transformer for Accelerated Multi-contrast MRI Reconstruction](https://mlsanity.com/paper/2201.10776/) (2022)
- [HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction](https://mlsanity.com/paper/2203.08213/) (2022)
- [Self-Consistent Recursive Diffusion Bridge for Medical Image Translation](https://mlsanity.com/paper/2405.06789/) (2024)
- [I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling](https://mlsanity.com/paper/2405.14022/) (2024)
- [BolT: Fused Window Transformers for fMRI Time Series Analysis](https://mlsanity.com/paper/2205.11578/) (2022)
- [One Model to Unite Them All: Personalized Federated Learning of Multi-Contrast MRI Synthesis](https://mlsanity.com/paper/2207.06509/) (2022)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [CyTran: A Cycle-Consistent Transformer with Multi-Level Consistency for Non-Contrast to Contrast CT Translation](https://mlsanity.com/paper/2110.06400/) (2021)
- [DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRI](https://mlsanity.com/paper/2307.09547/) (2023)
- [Learning Fourier-Constrained Diffusion Bridges for MRI Reconstruction](https://mlsanity.com/paper/2308.01096/) (2023)
- [A review and experimental evaluation of deep learning methods for MRI reconstruction](https://mlsanity.com/paper/2109.08618/) (2021)
- [Dual-Domain Self-Supervised Learning for Accelerated Non-Cartesian MRI Reconstruction](https://mlsanity.com/paper/2302.09244/) (2023)
- [One-shot Generative Prior in Hankel-k-space for Parallel Imaging Reconstruction](https://mlsanity.com/paper/2208.07181/) (2022)
- [LIT-Former: Linking In-plane and Through-plane Transformers for Simultaneous CT Image Denoising and Deblurring](https://mlsanity.com/paper/2302.10630/) (2023)

## Abstract (arXiv)

Supervised reconstruction models are characteristically trained on matched pairs of undersampled and fully-sampled data to capture an MRI prior, along with supervision regarding the imaging operator to enforce data consistency. To reduce supervision requirements, the recent deep image prior framework instead conjoins untrained MRI priors with the imaging operator during inference. Yet, canonical convolutional architectures are suboptimal in capturing long-range relationships, and priors based on randomly initialized networks may yield suboptimal performance. To address these limitations, here we introduce a novel unsupervised MRI reconstruction method based on zero-Shot Learned Adversarial TransformERs (SLATER). SLATER embodies a deep adversarial network with cross-attention transformers to map noise and latent variables onto coil-combined MR images. During pre-training, this unconditional network learns a high-quality MRI prior in an unsupervised generative modeling task. During inference, a zero-shot reconstruction is then performed by incorporating the imaging operator and optimizing the prior to maximize consistency to undersampled data. Comprehensive experiments on brain MRI datasets clearly demonstrate the superior performance of SLATER against state-of-the-art unsupervised methods.