# Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences

arXiv: https://arxiv.org/abs/1806.04066 · HTML: https://mlsanity.com/paper/1806.04066/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2018.

## Built on by (most cited first, 30 of 47)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Ultrasound Video Transformers for Cardiac Ejection Fraction Estimation](https://mlsanity.com/paper/2107.00977/) (2021)
- [DeepTag: An Unsupervised Deep Learning Method for Motion Tracking on Cardiac Tagging Magnetic Resonance Images](https://mlsanity.com/paper/2103.02772/) (2021)
- [Learning a Model-Driven Variational Network for Deformable Image Registration](https://mlsanity.com/paper/2105.12227/) (2021)
- [Generative Myocardial Motion Tracking via Latent Space Exploration with Biomechanics-informed Prior](https://mlsanity.com/paper/2206.03830/) (2022)
- [MulViMotion: Shape-aware 3D Myocardial Motion Tracking from Multi-View Cardiac MRI](https://mlsanity.com/paper/2208.00034/) (2022)
- [4D Myocardium Reconstruction with Decoupled Motion and Shape Model](https://mlsanity.com/paper/2308.14083/) (2023)
- [DeepMesh: Mesh-based Cardiac Motion Tracking using Deep Learning](https://mlsanity.com/paper/2309.14306/) (2023)
- [Towards Cardiac MRI Foundation Models: Comprehensive Visual-Tabular Representations for Whole-Heart Assessment and Beyond](https://mlsanity.com/paper/2504.13037/) (2025)
- [Cyclical Self-Supervision for Semi-Supervised Ejection Fraction Prediction from Echocardiogram Videos](https://mlsanity.com/paper/2210.11291/) (2022)
- [Cross-Modality Image Registration using a Training-Time Privileged Third Modality](https://mlsanity.com/paper/2207.12901/) (2022)
- [Mesh-based 3D Motion Tracking in Cardiac MRI using Deep Learning](https://mlsanity.com/paper/2209.02004/) (2022)
- [Whole Heart 3D+T Representation Learning Through Sparse 2D Cardiac MR Images](https://mlsanity.com/paper/2406.00329/) (2024)
- [Motion-compensated MR CINE reconstruction with reconstruction-driven motion estimation](https://mlsanity.com/paper/2302.02504/) (2023)
- [Aligning Multi-Sequence CMR Towards Fully Automated Myocardial Pathology Segmentation](https://mlsanity.com/paper/2302.03537/) (2023)
- [Modeling 3D cardiac contraction and relaxation with point cloud deformation networks](https://mlsanity.com/paper/2307.10927/) (2023)
- [End-to-End 4D Heart Mesh Recovery Across Full-Stack and Sparse Cardiac MRI](https://mlsanity.com/paper/2509.12090/) (2025)
- [CartiMorph: a framework for automated knee articular cartilage morphometrics](https://mlsanity.com/paper/2308.01981/) (2023)
- [Decoder-Only Image Registration](https://mlsanity.com/paper/2402.03585/) (2024)
- [Direct Cardiac Segmentation from Undersampled K-space Using Transformers](https://mlsanity.com/paper/2406.00192/) (2024)
- [Reciprocal Landmark Detection and Tracking with Extremely Few Annotations](https://mlsanity.com/paper/2101.11224/) (2021)
- [Fourier-Net+: Leveraging Band-Limited Representation for Efficient 3D Medical Image Registration](https://mlsanity.com/paper/2307.02997/) (2023)
- [Attention-aware non-rigid image registration for accelerated MR imaging](https://mlsanity.com/paper/2404.17621/) (2024)
- [Deep Statistic Shape Model for Myocardium Segmentation](https://mlsanity.com/paper/2207.10607/) (2022)
- [CineMyoPS: Segmenting Myocardial Pathologies from Cine Cardiac MR](https://mlsanity.com/paper/2507.02289/) (2025)
- [Bootstrap Representation Learning for Segmentation on Medical Volumes and Sequences](https://mlsanity.com/paper/2106.12153/) (2021)
- [Assessment of Left Atrium Motion Deformation Through Full Cardiac Cycle](https://mlsanity.com/paper/2405.17518/) (2024)
- [MBSS-T1: Model-Based Subject-Specific Self-Supervised Motion Correction for Robust Cardiac T1 Mapping](https://mlsanity.com/paper/2408.11992/) (2024)
- [UniReg: Conditional Unified Model for Medical Image Registration](https://mlsanity.com/paper/2503.12868/) (2025)
- [Multi-class motion-based semantic segmentation for ureteroscopy and laser lithotripsy](https://mlsanity.com/paper/2104.01268/) (2021)

## Abstract (arXiv)

Cardiac motion estimation and segmentation play important roles in quantitatively assessing cardiac function and diagnosing cardiovascular diseases. In this paper, we propose a novel deep learning method for joint estimation of motion and segmentation from cardiac MR image sequences. The proposed network consists of two branches: a cardiac motion estimation branch which is built on a novel unsupervised Siamese style recurrent spatial transformer network, and a cardiac segmentation branch that is based on a fully convolutional network. In particular, a joint multi-scale feature encoder is learned by optimizing the segmentation branch and the motion estimation branch simultaneously. This enables the weakly-supervised segmentation by taking advantage of features that are unsupervisedly learned in the motion estimation branch from a large amount of unannotated data. Experimental results using cardiac MRI images from 220 subjects show that the joint learning of both tasks is complementary and the proposed models outperform the competing methods significantly in terms of accuracy and speed.