# One Network to Solve Them All: A Sequential Multi-Task Joint Learning Network Framework for MR Imaging Pipeline

arXiv: https://arxiv.org/abs/2105.06653 · HTML: https://mlsanity.com/paper/2105.06653/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 14 May 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Connecting Image Denoising and High-Level Vision Tasks via Deep Learning](https://mlsanity.com/paper/1809.01826/) (2018)
- [Learning-based Optimization of the Under-sampling Pattern in MRI](https://mlsanity.com/paper/1901.01960/) (2019)
- [Self-Supervised Deep Active Accelerated MRI](https://mlsanity.com/paper/1901.04547/) (2019)
- [Joint CS-MRI Reconstruction and Segmentation with a Unified Deep Network](https://mlsanity.com/paper/1805.02165/) (2018)
- [A Segmentation-aware Deep Fusion Network for Compressed Sensing MRI](https://mlsanity.com/paper/1804.01210/) (2018)
- [Machine Friendly Machine Learning: Interpretation of Computed Tomography Without Image Reconstruction](https://mlsanity.com/paper/1812.01068/) (2018)
- [End-to-end Lung Nodule Detection in Computed Tomography](https://mlsanity.com/paper/1711.02074/) (2017)

## Built on by (most cited first, 5 of 5)

- [Learning Task-Specific Strategies for Accelerated MRI](https://mlsanity.com/paper/2304.12507/) (2023)
- [MOST: MR reconstruction Optimization for multiple downStream Tasks via continual learning](https://mlsanity.com/paper/2409.10394/) (2024)
- [R2C-GAN: Restore-to-Classify Generative Adversarial Networks for Blind X-Ray Restoration and COVID-19 Classification](https://mlsanity.com/paper/2209.14770/) (2022)
- [Learning Task-Specific Sampling Strategy for Sparse-View CT Reconstruction](https://mlsanity.com/paper/2409.01544/) (2024)
- [Information-Theoretic Optimization for Task-Adapted Compressed Sensing Magnetic Resonance Imaging](https://mlsanity.com/paper/2604.12709/) (2026)

## Abstract (arXiv)

Magnetic resonance imaging (MRI) acquisition, reconstruction, and segmentation are usually processed independently in the conventional practice of MRI workflow. It is easy to notice that there are significant relevances among these tasks and this procedure artificially cuts off these potential connections, which may lead to losing clinically important information for the final diagnosis. To involve these potential relations for further performance improvement, a sequential multi-task joint learning network model is proposed to train a combined end-to-end pipeline in a differentiable way, aiming at exploring the mutual influence among those tasks simultaneously. Our design consists of three cascaded modules: 1) deep sampling pattern learning module optimizes the k-space sampling pattern with predetermined sampling rate; 2) deep reconstruction module is dedicated to reconstructing MR images from the undersampled data using the learned sampling pattern; 3) deep segmentation module encodes MR images reconstructed from the previous module to segment the interested tissues. The proposed model retrieves the latently interactive and cyclic relations among those tasks, from which each task will be mutually beneficial. The proposed framework is verified on MRB dataset, which achieves superior performance on other SOTA methods in terms of both reconstruction and segmentation.