# End-to-End Sequential Sampling and Reconstruction for MRI

arXiv: https://arxiv.org/abs/2105.06460 · HTML: https://mlsanity.com/paper/2105.06460/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 60 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1704.02422/) (2017)
- [Image reconstruction by domain transform manifold learning](https://mlsanity.com/paper/1704.08841/) (2017)
- [End-to-End Variational Networks for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2004.06688/) (2020)
- [Deep learning for undersampled MRI reconstruction](https://mlsanity.com/paper/1709.02576/) (2017)
- [Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss](https://mlsanity.com/paper/1709.00753/) (2017)
- [RARE: Image Reconstruction using Deep Priors Learned without Ground Truth](https://mlsanity.com/paper/1912.05854/) (2019)
- [Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge](https://mlsanity.com/paper/2001.02518/) (2020)
- [Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging](https://mlsanity.com/paper/2010.14462/) (2020)
- [Reducing Uncertainty in Undersampled MRI Reconstruction with Active Acquisition](https://mlsanity.com/paper/1902.03051/) (2019)
- [Active MR k-space Sampling with Reinforcement Learning](https://mlsanity.com/paper/2007.10469/) (2020)
- [Experimental design for MRI by greedy policy search](https://mlsanity.com/paper/2010.16262/) (2020)
- [Learning-based Optimization of the Under-sampling Pattern in MRI](https://mlsanity.com/paper/1901.01960/) (2019)
- [B-spline Parameterized Joint Optimization of Reconstruction and K-space Trajectories (BJORK) for Accelerated 2D MRI](https://mlsanity.com/paper/2101.11369/) (2021)
- [Self-Supervised Deep Active Accelerated MRI](https://mlsanity.com/paper/1901.04547/) (2019)
- [PILOT: Physics-Informed Learned Optimized Trajectories for Accelerated MRI](https://mlsanity.com/paper/1909.05773/) (2019)
- [Extending LOUPE for K-space Under-sampling Pattern Optimization in Multi-coil MRI](https://mlsanity.com/paper/2007.14450/) (2020)
- [Physics-based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging](https://mlsanity.com/paper/1808.03571/) (2018)
- [Data-Driven Design for Fourier Ptychographic Microscopy](https://mlsanity.com/paper/1904.04175/) (2019)
- [SGD-Net: Efficient Model-Based Deep Learning with Theoretical Guarantees](https://mlsanity.com/paper/2101.09379/) (2021)
- [Joint learning of cartesian undersampling and reconstruction for accelerated MRI](https://mlsanity.com/paper/1905.09324/) (2019)
- [Fast Multi-class Dictionaries Learning with Geometrical Directions in MRI Reconstruction](https://mlsanity.com/paper/1503.02945/) (2015)

## Built on by (most cited first, 13 of 13)

- [Symbolic Learning to Optimize: Towards Interpretability and Scalability](https://mlsanity.com/paper/2203.06578/) (2022)
- [On learning adaptive acquisition policies for undersampled multi-coil MRI reconstruction](https://mlsanity.com/paper/2203.16392/) (2022)
- [Probabilistic Bayesian optimal experimental design using conditional normalizing flows](https://mlsanity.com/paper/2402.18337/) (2024)
- [JoJoNet: Joint-contrast and Joint-sampling-and-reconstruction Network for Multi-contrast MRI](https://mlsanity.com/paper/2210.12548/) (2022)
- [Single-pass Object-adaptive Data Undersampling and Reconstruction for MRI](https://mlsanity.com/paper/2111.09212/) (2021)
- [End-to-end Adaptive Dynamic Subsampling and Reconstruction for Cardiac MRI](https://mlsanity.com/paper/2403.10346/) (2024)
- [Adaptive Selection of Sampling-Reconstruction in Fourier Compressed Sensing](https://mlsanity.com/paper/2409.11738/) (2024)
- [Active Phase-Encode Selection for Slice-Specific Fast MR Scanning Using a Transformer-Based Deep Reinforcement Learning Framework](https://mlsanity.com/paper/2203.05756/) (2022)
- [A Path Towards Clinical Adaptation of Accelerated MRI](https://mlsanity.com/paper/2208.12835/) (2022)
- [Learning to sample in Cartesian MRI](https://mlsanity.com/paper/2312.04327/) (2023)
- [A Note on Machine Learning Approach for Computational Imaging](https://mlsanity.com/paper/2202.11883/) (2022)
- [L2SR: Learning to Sample and Reconstruct for Accelerated MRI via Reinforcement Learning](https://mlsanity.com/paper/2212.02190/) (2022)
- [Reinforcement Learning of Adaptive Acquisition Policies for Inverse Problems](https://mlsanity.com/paper/2407.07794/) (2024)

## Abstract (arXiv)

Accelerated MRI shortens acquisition time by subsampling in the measurement kappa-space. Recovering a high-fidelity anatomical image from subsampled measurements requires close cooperation between two components: (1) a sampler that chooses the subsampling pattern and (2) a reconstructor that recovers images from incomplete measurements. In this paper, we leverage the sequential nature of MRI measurements, and propose a fully differentiable framework that jointly learns a sequential sampling policy simultaneously with a reconstruction strategy. This co-designed framework is able to adapt during acquisition in order to capture the most informative measurements for a particular target. Experimental results on the fastMRI knee dataset demonstrate that the proposed approach successfully utilizes intermediate information during the sampling process to boost reconstruction performance. In particular, our proposed method can outperform the current state-of-the-art learned kappa-space sampling baseline on over 96% of test samples. We also investigate the individual and collective benefits of the sequential sampling and co-design strategies.