# DeepRecon: Joint 2D Cardiac Segmentation and 3D Volume Reconstruction via A Structure-Specific Generative Method

arXiv: https://arxiv.org/abs/2206.07163 · HTML: https://mlsanity.com/paper/2206.07163/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [In-Domain GAN Inversion for Real Image Editing](https://mlsanity.com/paper/2004.00049/) (2020)
- [Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization](https://mlsanity.com/paper/2104.05833/) (2021)
- [UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation](https://mlsanity.com/paper/2107.00781/) (2021)
- [A Data-scalable Transformer for Medical Image Segmentation: Architecture, Model Efficiency, and Benchmark](https://mlsanity.com/paper/2203.00131/) (2022)
- [DeepTag: An Unsupervised Deep Learning Method for Motion Tracking on Cardiac Tagging Magnetic Resonance Images](https://mlsanity.com/paper/2103.02772/) (2021)
- [TransFusion: Multi-view Divergent Fusion for Medical Image Segmentation with Transformers](https://mlsanity.com/paper/2203.10726/) (2022)
- [3D High-Resolution Cardiac Segmentation Reconstruction from 2D Views using Conditional Variational Autoencoders](https://mlsanity.com/paper/1902.11000/) (2019)

## Built on by (most cited first, 17 of 17)

- [Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentation](https://mlsanity.com/paper/2306.02416/) (2023)
- [Show and Segment: Universal Medical Image Segmentation via In-Context Learning](https://mlsanity.com/paper/2503.19359/) (2025)
- [Layout Agnostic Scene Text Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2406.01062/) (2024)
- [DeFormer: Integrating Transformers with Deformable Models for 3D Shape Abstraction from a Single Image](https://mlsanity.com/paper/2309.12594/) (2023)
- [Dealing With Heterogeneous 3D MR Knee Images: A Federated Few-Shot Learning Method With Dual Knowledge Distillation](https://mlsanity.com/paper/2303.14357/) (2023)
- [DMCVR: Morphology-Guided Diffusion Model for 3D Cardiac Volume Reconstruction](https://mlsanity.com/paper/2308.09223/) (2023)
- [LUCAS: Layered Universal Codec Avatars](https://mlsanity.com/paper/2502.19739/) (2025)
- [Deep Deformable Models: Learning 3D Shape Abstractions with Part Consistency](https://mlsanity.com/paper/2309.01035/) (2023)
- [From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction](https://mlsanity.com/paper/2505.03599/) (2025)
- [Instantaneous Perception of Moving Objects in 3D](https://mlsanity.com/paper/2405.02781/) (2024)
- [Latent Interpolation Learning Using Diffusion Models for Cardiac Volume Reconstruction](https://mlsanity.com/paper/2508.13826/) (2025)
- [From 2D to 3D, Deep Learning-based Shape Reconstruction in Magnetic Resonance Imaging: A Review](https://mlsanity.com/paper/2510.01296/) (2025)
- [Resolving Inconsistent Semantics in Multi-Dataset Image Segmentation](https://mlsanity.com/paper/2409.09893/) (2024)
- [Large Sign Language Models: Toward 3D American Sign Language Translation](https://mlsanity.com/paper/2511.08535/) (2025)
- [LiverUSRecon: Automatic 3D Reconstruction and Volumetry of the Liver with a Few Partial Ultrasound Scans](https://mlsanity.com/paper/2406.19336/) (2024)
- [Med-2D SegNet: A Light Weight Deep Neural Network for Medical 2D Image Segmentation](https://mlsanity.com/paper/2504.14715/) (2025)
- [Personalized 4D Whole Heart Geometry Reconstruction from Cine MRI for Cardiac Digital Twins](https://mlsanity.com/paper/2507.15203/) (2025)

## Abstract (arXiv)

Joint 2D cardiac segmentation and 3D volume reconstruction are fundamental to building statistical cardiac anatomy models and understanding functional mechanisms from motion patterns. However, due to the low through-plane resolution of cine MR and high inter-subject variance, accurately segmenting cardiac images and reconstructing the 3D volume are challenging. In this study, we propose an end-to-end latent-space-based framework, DeepRecon, that generates multiple clinically essential outcomes, including accurate image segmentation, synthetic high-resolution 3D image, and 3D reconstructed volume. Our method identifies the optimal latent representation of the cine image that contains accurate semantic information for cardiac structures. In particular, our model jointly generates synthetic images with accurate semantic information and segmentation of the cardiac structures using the optimal latent representation. We further explore downstream applications of 3D shape reconstruction and 4D motion pattern adaptation by the different latent-space manipulation strategies.The simultaneously generated high-resolution images present a high interpretable value to assess the cardiac shape and motion.Experimental results demonstrate the effectiveness of our approach on multiple fronts including 2D segmentation, 3D reconstruction, downstream 4D motion pattern adaption performance.