# HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction

arXiv: https://arxiv.org/abs/2203.08213 · HTML: https://mlsanity.com/paper/2203.08213/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 65 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Deep Equilibrium Models](https://mlsanity.com/paper/1909.01377/) (2019)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [Deep Learning Techniques for Inverse Problems in Imaging](https://mlsanity.com/paper/2005.06001/) (2020)
- [End-to-End Variational Networks for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2004.06688/) (2020)
- [Deep Equilibrium Architectures for Inverse Problems in Imaging](https://mlsanity.com/paper/2102.07944/) (2021)
- [Video Super-Resolution Transformer](https://mlsanity.com/paper/2106.06847/) (2021)
- [MISSFormer: An Effective Medical Image Segmentation Transformer](https://mlsanity.com/paper/2109.07162/) (2021)
- [Deep learning for undersampled MRI reconstruction](https://mlsanity.com/paper/1709.02576/) (2017)
- [Framing U-Net via Deep Convolutional Framelets: Application to Sparse-view CT](https://mlsanity.com/paper/1708.08333/) (2017)
- [Unsupervised MRI Reconstruction via Zero-Shot Learned Adversarial Transformers](https://mlsanity.com/paper/2105.08059/) (2021)

## Built on by (most cited first, 30 of 30)

- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [Fill the K-Space and Refine the Image: Prompting for Dynamic and Multi-Contrast MRI Reconstruction](https://mlsanity.com/paper/2309.13839/) (2023)
- [Scaling Laws For Deep Learning Based Image Reconstruction](https://mlsanity.com/paper/2209.13435/) (2022)
- [Progressive Divide-and-Conquer via Subsampling Decomposition for Accelerated MRI](https://mlsanity.com/paper/2403.10064/) (2024)
- [Robustness of Deep Learning for Accelerated MRI: Benefits of Diverse Training Data](https://mlsanity.com/paper/2312.10271/) (2023)
- [Deep Learning for Accelerated and Robust MRI Reconstruction: a Review](https://mlsanity.com/paper/2404.15692/) (2024)
- [Analyzing the Sample Complexity of Self-Supervised Image Reconstruction Methods](https://mlsanity.com/paper/2305.19079/) (2023)
- [MambaRecon: MRI Reconstruction with Structured State Space Models](https://mlsanity.com/paper/2409.12401/) (2024)
- [QN-Mixer: A Quasi-Newton MLP-Mixer Model for Sparse-View CT Reconstruction](https://mlsanity.com/paper/2402.17951/) (2024)
- [Loop Unrolled Shallow Equilibrium Regularizer (LUSER) -- A Memory-Efficient Inverse Problem Solver](https://mlsanity.com/paper/2210.04987/) (2022)
- [CL-MRI: Self-Supervised Contrastive Learning to Improve the Accuracy of Undersampled MRI Reconstruction](https://mlsanity.com/paper/2306.00530/) (2023)
- [CAMP-Net: Consistency-Aware Multi-Prior Network for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2306.11238/) (2023)
- [Score-based Generative Priors Guided Model-driven Network for MRI Reconstruction](https://mlsanity.com/paper/2405.02958/) (2024)
- [MCU-Net: A Multi-prior Collaborative Deep Unfolding Network with Gates-controlled Spatial Attention for Accelerated MR Image Reconstruction](https://mlsanity.com/paper/2402.03383/) (2024)
- [Joint Edge Optimization Deep Unfolding Network for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2405.05564/) (2024)
- [LoFi: Neural Local Fields for Scalable Image Reconstruction](https://mlsanity.com/paper/2411.04995/) (2024)
- [PETAL: Physics Emulation Through Averaged Linearizations for Solving Inverse Problems](https://mlsanity.com/paper/2305.11056/) (2023)
- [JotlasNet: Joint Tensor Low-Rank and Attention-based Sparse Unrolling Network for Accelerating Dynamic MRI](https://mlsanity.com/paper/2502.11749/) (2025)
- [Efficient Noise Calculation in Deep Learning-based MRI Reconstructions](https://mlsanity.com/paper/2505.02007/) (2025)
- [When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks](https://mlsanity.com/paper/2606.04476/) (2026)
- [Experimental Design for Multi-Channel Imaging via Task-Driven Feature Selection](https://mlsanity.com/paper/2210.06891/) (2022)
- [k-t CLAIR: Self-Consistency Guided Multi-Prior Learning for Dynamic Parallel MR Image Reconstruction](https://mlsanity.com/paper/2310.11050/) (2023)
- [UnWave-Net: Unrolled Wavelet Network for Compton Tomography Image Reconstruction](https://mlsanity.com/paper/2406.03413/) (2024)
- [SGD Jittering: A Training Strategy for Robust and Accurate Model-Based Architectures](https://mlsanity.com/paper/2410.14667/) (2024)
- [A Master Class on Reproducibility: A Student Hackathon on Advanced MRI Reconstruction Methods](https://mlsanity.com/paper/2601.18314/) (2026)
- [Revisiting Global Token Mixing in Task-Dependent MRI Restoration: Insights from Minimal Gated CNN Baselines](https://mlsanity.com/paper/2603.01449/) (2026)
- [Transferable Optimization Network for Cross-Domain Image Reconstruction](https://mlsanity.com/paper/2603.07831/) (2026)
- [UEPS: Robust and Efficient MRI Reconstruction](https://mlsanity.com/paper/2603.18572/) (2026)
- [MosaicMRI: A Diverse Dataset and Benchmark for Raw Musculoskeletal MRI](https://mlsanity.com/paper/2604.11762/) (2026)
- [Are Natural-Domain Foundation Models Effective for Accelerated Cardiac MRI Reconstruction?](https://mlsanity.com/paper/2604.22557/) (2026)

## Abstract (arXiv)

In accelerated MRI reconstruction, the anatomy of a patient is recovered from a set of under-sampled and noisy measurements. Deep learning approaches have been proven to be successful in solving this ill-posed inverse problem and are capable of producing very high quality reconstructions. However, current architectures heavily rely on convolutions, that are content-independent and have difficulties modeling long-range dependencies in images. Recently, Transformers, the workhorse of contemporary natural language processing, have emerged as powerful building blocks for a multitude of vision tasks. These models split input images into non-overlapping patches, embed the patches into lower-dimensional tokens and utilize a self-attention mechanism that does not suffer from the aforementioned weaknesses of convolutional architectures. However, Transformers incur extremely high compute and memory cost when 1) the input image resolution is high and 2) when the image needs to be split into a large number of patches to preserve fine detail information, both of which are typical in low-level vision problems such as MRI reconstruction, having a compounding effect. To tackle these challenges, we propose HUMUS-Net, a hybrid architecture that combines the beneficial implicit bias and efficiency of convolutions with the power of Transformer blocks in an unrolled and multi-scale network. HUMUS-Net extracts high-resolution features via convolutional blocks and refines low-resolution features via a novel Transformer-based multi-scale feature extractor. Features from both levels are then synthesized into a high-resolution output reconstruction. Our network establishes new state of the art on the largest publicly available MRI dataset, the fastMRI dataset. We further demonstrate the performance of HUMUS-Net on two other popular MRI datasets and perform fine-grained ablation studies to validate our design.