# A Deep-Learning Approach For Direct Whole-Heart Mesh Reconstruction

arXiv: https://arxiv.org/abs/2102.07899 · HTML: https://mlsanity.com/paper/2102.07899/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 61 references that are in the map.
- First posted on arXiv 16 Feb 2021.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Geometric deep learning: going beyond Euclidean data](https://mlsanity.com/paper/1611.08097/) (2016)
- [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019)
- [The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge](https://mlsanity.com/paper/1912.01054/) (2019)
- [Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation](https://mlsanity.com/paper/1908.01491/) (2019)
- [Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge](https://mlsanity.com/paper/1902.07880/) (2019)
- [Neural Mesh Flow: 3D Manifold Mesh Generation via Diffeomorphic Flows](https://mlsanity.com/paper/2007.10973/) (2020)
- [Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge](https://mlsanity.com/paper/1802.10508/) (2018)
- [Voxel2Mesh: 3D Mesh Model Generation from Volumetric Data](https://mlsanity.com/paper/1912.03681/) (2019)
- [MultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learning](https://mlsanity.com/paper/1904.08492/) (2019)
- [Image2Mesh: A Learning Framework for Single Image 3D Reconstruction](https://mlsanity.com/paper/1711.10669/) (2017)
- [An attempt at beating the 3D U-Net](https://mlsanity.com/paper/1908.02182/) (2019)
- [DeepOrganNet: On-the-Fly Reconstruction and Visualization of 3D / 4D Lung Models from Single-View Projections by Deep Deformation Network](https://mlsanity.com/paper/1907.09375/) (2019)
- [PC-U Net: Learning to Jointly Reconstruct and Segment the Cardiac Walls in 3D from CT Data](https://mlsanity.com/paper/2008.08194/) (2020)

## Built on by (most cited first, 30 of 47)

- [Vox2Cortex: Fast Explicit Reconstruction of Cortical Surfaces from 3D MRI Scans with Geometric Deep Neural Networks](https://mlsanity.com/paper/2203.09446/) (2022)
- [Learning Whole Heart Mesh Generation From Patient Images For Computational Simulations](https://mlsanity.com/paper/2203.10517/) (2022)
- [Learning Reduced-Order Models for Cardiovascular Simulations with Graph Neural Networks](https://mlsanity.com/paper/2303.07310/) (2023)
- [4D Myocardium Reconstruction with Decoupled Motion and Shape Model](https://mlsanity.com/paper/2308.14083/) (2023)
- [Neural deformation fields for template-based reconstruction of cortical surfaces from MRI](https://mlsanity.com/paper/2401.12938/) (2024)
- [Shape of my heart: Cardiac models through learned signed distance functions](https://mlsanity.com/paper/2308.16568/) (2023)
- [Improved 3D Whole Heart Geometry from Sparse CMR Slices](https://mlsanity.com/paper/2408.07532/) (2024)
- [Explicit Differentiable Slicing and Global Deformation for Cardiac Mesh Reconstruction](https://mlsanity.com/paper/2409.02070/) (2024)
- [End-to-End 4D Heart Mesh Recovery Across Full-Stack and Sparse Cardiac MRI](https://mlsanity.com/paper/2509.12090/) (2025)
- [DeformCL: Learning Deformable Centerline Representation for Vessel Extraction in 3D Medical Image](https://mlsanity.com/paper/2506.05820/) (2025)
- [3D Coronary Vessel Reconstruction from Bi-Plane Angiography using Graph Convolutional Networks](https://mlsanity.com/paper/2302.14795/) (2023)
- [Abdominal organ segmentation via deep diffeomorphic mesh deformations](https://mlsanity.com/paper/2306.15515/) (2023)
- [LinFlo-Net: A two-stage deep learning method to generate simulation ready meshes of the heart](https://mlsanity.com/paper/2310.20065/) (2023)
- [From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction](https://mlsanity.com/paper/2505.03599/) (2025)
- [Cardiac Digital Twins at Scale from MRI: Open Tools and Representative Models from ~55000 UK Biobank Participants](https://mlsanity.com/paper/2505.21019/) (2025)
- [Neural Implicit Heart Coordinates: 3D cardiac shape reconstruction from sparse segmentations](https://mlsanity.com/paper/2512.19316/) (2025)
- [Modeling and hexahedral meshing of cerebral arterial networks from centerlines](https://mlsanity.com/paper/2201.08279/) (2022)
- [Attention-based Shape-Deformation Networks for Artifact-Free Geometry Reconstruction of Lumbar Spine from MR Images](https://mlsanity.com/paper/2404.00231/) (2024)
- [CineMesh4D: Personalized 4D Whole Heart Reconstruction from Sparse Cine MRI](https://mlsanity.com/paper/2605.13994/) (2026)
- [Pairwise-Constrained Implicit Functions for 3D Human Heart Modelling](https://mlsanity.com/paper/2307.08716/) (2023)
- [AI-Powered Automated Model Construction for Patient-Specific CFD Simulations of Aortic Flows](https://mlsanity.com/paper/2503.12515/) (2025)
- [From 2D to 3D, Deep Learning-based Shape Reconstruction in Magnetic Resonance Imaging: A Review](https://mlsanity.com/paper/2510.01296/) (2025)
- [Spatial Correspondence between Graph Neural Network-Segmented Images](https://mlsanity.com/paper/2303.06550/) (2023)
- [Intelligent Generation of Graphical Game Assets: A Conceptual Framework and Systematic Review of the State of the Art](https://mlsanity.com/paper/2311.10129/) (2023)
- [Image2Flow: A hybrid image and graph convolutional neural network for rapid patient-specific pulmonary artery segmentation and CFD flow field calculation from 3D cardiac MRI data](https://mlsanity.com/paper/2402.18236/) (2024)
- [Graph Convolutional Neural Networks for Automated Echocardiography View Recognition: A Holistic Approach](https://mlsanity.com/paper/2402.19062/) (2024)
- [Robust automated calcification meshing for biomechanical cardiac digital twins](https://mlsanity.com/paper/2403.04998/) (2024)
- [MorphiNet: A Graph Subdivision Network for Adaptive Bi-ventricle Surface Reconstruction](https://mlsanity.com/paper/2412.10985/) (2024)
- [SeqSeg: Learning Local Segments for Automatic Vascular Model Construction](https://mlsanity.com/paper/2501.15712/) (2025)
- [Relational Anatomical Supervision for Accurate 3D Multi-Chamber Cardiac Mesh Reconstruction](https://mlsanity.com/paper/2503.07874/) (2025)

## Abstract (arXiv)

Automated construction of surface geometries of cardiac structures from volumetric medical images is important for a number of clinical applications. While deep-learning-based approaches have demonstrated promising reconstruction precision, these approaches have mostly focused on voxel-wise segmentation followed by surface reconstruction and post-processing techniques. However, such approaches suffer from a number of limitations including disconnected regions or incorrect surface topology due to erroneous segmentation and stair-case artifacts due to limited segmentation resolution. We propose a novel deep-learning-based approach that directly predicts whole heart surface meshes from volumetric CT and MR image data. Our approach leverages a graph convolutional neural network to predict deformation on mesh vertices from a pre-defined mesh template to reconstruct multiple anatomical structures in a 3D image volume. Our method demonstrated promising performance of generating whole heart reconstructions with as good or better accuracy than prior deep-learning-based methods on both CT and MR data. Furthermore, by deforming a template mesh, our method can generate whole heart geometries with better anatomical consistency and produce high-resolution geometries from lower resolution input image data. Our method was also able to produce temporally consistent surface mesh predictions for heart motion from CT or MR cine sequences, and therefore can potentially be applied for efficiently constructing 4D whole heart dynamics. Our code and pre-trained networks are available at https://github.com/fkong7/MeshDeformNet