# Deep End-to-end Adaptive k-Space Sampling, Reconstruction, and Registration for Dynamic MRI

arXiv: https://arxiv.org/abs/2411.18249 · HTML: https://mlsanity.com/paper/2411.18249/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 69 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Builds on (most cited first, 30 of 33)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [MoDL: Model Based Deep Learning Architecture for Inverse Problems](https://mlsanity.com/paper/1712.02862/) (2017)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [End-to-End Variational Networks for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2004.06688/) (2020)
- [Time-Dependent Deep Image Prior for Dynamic MRI](https://mlsanity.com/paper/1910.01684/) (2019)
- [Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge](https://mlsanity.com/paper/2001.02518/) (2020)
- [AI-based Reconstruction for Fast MRI -- A Systematic Review and Meta-analysis](https://mlsanity.com/paper/2112.12744/) (2021)
- [Recurrent Variational Network: A Deep Learning Inverse Problem Solver applied to the task of Accelerated MRI Reconstruction](https://mlsanity.com/paper/2111.09639/) (2021)
- [Learning-based Optimization of the Under-sampling Pattern in MRI](https://mlsanity.com/paper/1901.01960/) (2019)
- [Extending LOUPE for K-space Under-sampling Pattern Optimization in Multi-coil MRI](https://mlsanity.com/paper/2007.14450/) (2020)
- [CMRxRecon: An open cardiac MRI dataset for the competition of accelerated image reconstruction](https://mlsanity.com/paper/2309.10836/) (2023)
- [Medical Image Registration via Neural Fields](https://mlsanity.com/paper/2206.03111/) (2022)
- [Multi-Coil MRI Reconstruction Challenge -- Assessing Brain MRI Reconstruction Models and their Generalizability to Varying Coil Configurations](https://mlsanity.com/paper/2011.07952/) (2020)
- [Self-Supervised Physics-Based Deep Learning MRI Reconstruction Without Fully-Sampled Data](https://mlsanity.com/paper/1910.09116/) (2019)
- [The state-of-the-art in Cardiac MRI Reconstruction: Results of the CMRxRecon Challenge in MICCAI 2023](https://mlsanity.com/paper/2404.01082/) (2024)
- [On learning adaptive acquisition policies for undersampled multi-coil MRI reconstruction](https://mlsanity.com/paper/2203.16392/) (2022)
- [A review and experimental evaluation of deep learning methods for MRI reconstruction](https://mlsanity.com/paper/2109.08618/) (2021)
- [On Retrospective k-space Subsampling schemes For Deep MRI Reconstruction](https://mlsanity.com/paper/2301.08365/) (2023)
- [vSHARP: variable Splitting Half-quadratic Admm algorithm for Reconstruction of inverse-Problems](https://mlsanity.com/paper/2309.09954/) (2023)
- [Enhancing Label-Driven Deep Deformable Image Registration with Local Distance Metrics for State-of-the-Art Cardiac Motion Tracking](https://mlsanity.com/paper/1812.01859/) (2018)
- [Deep Cardiac MRI Reconstruction with ADMM](https://mlsanity.com/paper/2310.06628/) (2023)
- [CMRxRecon2024: A Multi-Modality, Multi-View K-Space Dataset Boosting Universal Machine Learning for Accelerated Cardiac MRI](https://mlsanity.com/paper/2406.19043/) (2024)
- [Variational Multi-Task MRI Reconstruction: Joint Reconstruction, Registration and Super-Resolution](https://mlsanity.com/paper/1908.05911/) (2019)
- [Joint Supervised and Self-supervised Learning for MRI Reconstruction](https://mlsanity.com/paper/2311.15856/) (2023)

## Abstract (arXiv)

Dynamic MRI enables a range of clinical applications, including cardiac function assessment, organ motion tracking, and radiotherapy guidance. However, fully sampling the dynamic k-space data is often infeasible due to time constraints and physiological motion such as respiratory and cardiac motion. This necessitates undersampling, which degrades the quality of reconstructed images. Poor image quality not only hinders visualization but also impairs the estimation of deformation fields, crucial for registering dynamic (moving) images to a static reference image. This registration enables tasks such as motion correction, treatment planning, and quantitative analysis in applications like cardiac imaging and MR-guided radiotherapy. To overcome the challenges posed by undersampling and motion, we introduce an end-to-end deep learning (DL) framework that integrates adaptive dynamic k-space sampling, reconstruction, and registration. Our approach begins with a DL-based adaptive sampling strategy, optimizing dynamic k-space acquisition to capture the most relevant data for each specific case. This is followed by a DL-based reconstruction module that produces images optimized for accurate deformation field estimation from the undersampled moving data. Finally, a registration module estimates the deformation fields aligning the reconstructed dynamic images with a static reference. The proposed framework is independent of specific reconstruction and registration modules allowing for plug-and-play integration of these components. The entire framework is jointly trained using a combination of supervised and unsupervised loss functions, enabling end-to-end optimization for improved performance across all components. Through controlled experiments and ablation studies, we validate each component, demonstrating that each choice contributes to robust motion estimation from undersampled dynamic data.