# Deep Implicit Statistical Shape Models for 3D Medical Image Delineation

arXiv: https://arxiv.org/abs/2104.02847 · HTML: https://mlsanity.com/paper/2104.02847/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 7 Apr 2021.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge](https://mlsanity.com/paper/1912.01054/) (2019)
- [Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation](https://mlsanity.com/paper/1705.08302/) (2017)
- [Superhuman Accuracy on the SNEMI3D Connectomics Challenge](https://mlsanity.com/paper/1706.00120/) (2017)
- [Voxel2Mesh: 3D Mesh Model Generation from Volumetric Data](https://mlsanity.com/paper/1912.03681/) (2019)
- [DeepSSM: A Deep Learning Framework for Statistical Shape Modeling from Raw Images](https://mlsanity.com/paper/1810.00111/) (2018)
- [An Artificial Agent for Robust Image Registration](https://mlsanity.com/paper/1611.10336/) (2016)
- [Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search](https://mlsanity.com/paper/2004.08426/) (2020)
- [Automatic Liver Segmentation Using an Adversarial Image-to-Image Network](https://mlsanity.com/paper/1707.08037/) (2017)
- [Co-Heterogeneous and Adaptive Segmentation from Multi-Source and Multi-Phase CT Imaging Data: A Study on Pathological Liver and Lesion Segmentation](https://mlsanity.com/paper/2005.13201/) (2020)
- [Progressive and Multi-Path Holistically Nested Neural Networks for Pathological Lung Segmentation from CT Images](https://mlsanity.com/paper/1706.03702/) (2017)
- [User-Guided Domain Adaptation for Rapid Annotation from User Interactions: A Study on Pathological Liver Segmentation](https://mlsanity.com/paper/2009.02455/) (2020)
- [End-to-End Adversarial Shape Learning for Abdomen Organ Deep Segmentation](https://mlsanity.com/paper/1910.06474/) (2019)

## Built on by (most cited first, 26 of 26)

- [MedShapeNet -- A Large-Scale Dataset of 3D Medical Shapes for Computer Vision](https://mlsanity.com/paper/2308.16139/) (2023)
- [Efficient Deformable Tissue Reconstruction via Orthogonal Neural Plane](https://mlsanity.com/paper/2312.15253/) (2023)
- [SDF4CHD: Generative Modeling of Cardiac Anatomies with Congenital Heart Defects](https://mlsanity.com/paper/2311.00332/) (2023)
- [4D Myocardium Reconstruction with Decoupled Motion and Shape Model](https://mlsanity.com/paper/2308.14083/) (2023)
- [Neural Deformable Models for 3D Bi-Ventricular Heart Shape Reconstruction and Modeling from 2D Sparse Cardiac Magnetic Resonance Imaging](https://mlsanity.com/paper/2307.07693/) (2023)
- [Improved post-hoc probability calibration for out-of-domain MRI segmentation](https://mlsanity.com/paper/2208.02870/) (2022)
- [SwIPE: Efficient and Robust Medical Image Segmentation with Implicit Patch Embeddings](https://mlsanity.com/paper/2307.12429/) (2023)
- [From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction](https://mlsanity.com/paper/2505.03599/) (2025)
- [MIRACLE3D: Memory-efficient Integrated Robust Approach for Continual Learning on Point Clouds via Shape Model Construction](https://mlsanity.com/paper/2410.06418/) (2024)
- [Multi-Target Landmark Detection with Incomplete Images via Reinforcement Learning and Shape Prior](https://mlsanity.com/paper/2301.05392/) (2023)
- [On the Localization of Ultrasound Image Slices within Point Distribution Models](https://mlsanity.com/paper/2309.00372/) (2023)
- [MASSM: An End-to-End Deep Learning Framework for Multi-Anatomy Statistical Shape Modeling Directly From Images](https://mlsanity.com/paper/2403.11008/) (2024)
- [DCSM 2.0: Deep Conditional Shape Models for Data Efficient Segmentation](https://mlsanity.com/paper/2407.00186/) (2024)
- [Large Intestine 3D Shape Refinement Using Point Diffusion Models for Digital Phantom Generation](https://mlsanity.com/paper/2309.08289/) (2023)
- [Q2A: Querying Implicit Fully Continuous Feature Pyramid to Align Features for Medical Image Segmentation](https://mlsanity.com/paper/2404.09472/) (2024)
- [Learning Volumetric Neural Deformable Models to Recover 3D Regional Heart Wall Motion from Multi-Planar Tagged MRI](https://mlsanity.com/paper/2411.15233/) (2024)
- [Uncertainty-aware retinal layer segmentation in OCT through probabilistic signed distance functions](https://mlsanity.com/paper/2412.04935/) (2024)
- [MiShape: 3D Shape Modelling of Mitochondria in Microscopy](https://mlsanity.com/paper/2303.01546/) (2023)
- [Learning to In-paint: Domain Adaptive Shape Completion for 3D Organ Segmentation](https://mlsanity.com/paper/2308.08775/) (2023)
- [On the Viability of Semi-Supervised Segmentation Methods for Statistical Shape Modeling](https://mlsanity.com/paper/2407.15260/) (2024)
- [Multi-Disease-Aware Training Strategy for Cardiac MR Image Segmentation](https://mlsanity.com/paper/2503.17896/) (2025)
- [NUDF: Neural Unsigned Distance Fields for high resolution 3D medical image segmentation](https://mlsanity.com/paper/2504.18344/) (2025)
- [Template-Guided Reconstruction of Pulmonary Segments with Neural Implicit Functions](https://mlsanity.com/paper/2505.08919/) (2025)
- [Convolutional Feature Noise Reduction for 2D Cardiac MR Image Segmentation](https://mlsanity.com/paper/2511.22983/) (2025)
- [UCAD: Uncertainty-guided Contour-aware Displacement for semi-supervised medical image segmentation](https://mlsanity.com/paper/2601.17366/) (2026)
- [Landmark-free Assessment of Lower-limb Alignment with Implicit Neural Shape Functions from Knee Radiographs](https://mlsanity.com/paper/2606.15250/) (2026)

## Abstract (arXiv)

3D delineation of anatomical structures is a cardinal goal in medical imaging analysis. Prior to deep learning, statistical shape models that imposed anatomical constraints and produced high quality surfaces were a core technology. Prior to deep learning, statistical shape models that imposed anatomical constraints and produced high quality surfaces were a core technology. Today fully-convolutional networks (FCNs), while dominant, do not offer these capabilities. We present deep implicit statistical shape models (DISSMs), a new approach to delineation that marries the representation power of convolutional neural networks (CNNs) with the robustness of SSMs. DISSMs use a deep implicit surface representation to produce a compact and descriptive shape latent space that permits statistical models of anatomical variance. To reliably fit anatomically plausible shapes to an image, we introduce a novel rigid and non-rigid pose estimation pipeline that is modelled as a Markov decision process(MDP). We outline a training regime that includes inverted episodic training and a deep realization of marginal space learning (MSL). Intra-dataset experiments on the task of pathological liver segmentation demonstrate that DISSMs can perform more robustly than three leading FCN models, including nnU-Net: reducing the mean Hausdorff distance (HD) by 7.7-14.3mm and improving the worst case Dice-Sorensen coefficient (DSC) by 1.2-2.3%. More critically, cross-dataset experiments on a dataset directly reflecting clinical deployment scenarios demonstrate that DISSMs improve the mean DSC and HD by 3.5-5.9% and 12.3-24.5mm, respectively, and the worst-case DSC by 5.4-7.3%. These improvements are over and above any benefits from representing delineations with high-quality surface.