# From Images to Probabilistic Anatomical Shapes: A Deep Variational Bottleneck Approach

arXiv: https://arxiv.org/abs/2205.06862 · HTML: https://mlsanity.com/paper/2205.06862/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [DeepSSM: A Deep Learning Framework for Statistical Shape Modeling from Raw Images](https://mlsanity.com/paper/1810.00111/) (2018)
- [DeepSSM: A Blueprint for Image-to-Shape Deep Learning Models](https://mlsanity.com/paper/2110.07152/) (2021)
- [Uncertain-DeepSSM: From Images to Probabilistic Shape Models](https://mlsanity.com/paper/2007.06516/) (2020)
- [Temporal HeartNet: Towards Human-Level Automatic Analysis of Fetal Cardiac Screening Video](https://mlsanity.com/paper/1707.00665/) (2017)
- [Uncertainty Quantification in CNN-Based Surface Prediction Using Shape Priors](https://mlsanity.com/paper/1807.11272/) (2018)
- [Deep Learning for End-to-End Atrial Fibrillation Recurrence Estimation](https://mlsanity.com/paper/1810.00475/) (2018)

## Built on by (most cited first, 17 of 17)

- [Point2SSM: Learning Morphological Variations of Anatomies from Point Cloud](https://mlsanity.com/paper/2305.14486/) (2023)
- [Mesh2SSM: From Surface Meshes to Statistical Shape Models of Anatomy](https://mlsanity.com/paper/2305.07805/) (2023)
- [S3M: Scalable Statistical Shape Modeling through Unsupervised Correspondences](https://mlsanity.com/paper/2304.07515/) (2023)
- [Fully Bayesian VIB-DeepSSM](https://mlsanity.com/paper/2305.05797/) (2023)
- [Can point cloud networks learn statistical shape models of anatomies?](https://mlsanity.com/paper/2305.05610/) (2023)
- [Image2SSM: Reimagining Statistical Shape Models from Images with Radial Basis Functions](https://mlsanity.com/paper/2305.11946/) (2023)
- [Point2SSM++: Self-Supervised Learning of Anatomical Shape Models from Point Clouds](https://mlsanity.com/paper/2405.09707/) (2024)
- [ADASSM: Adversarial Data Augmentation in Statistical Shape Models From Images](https://mlsanity.com/paper/2307.03273/) (2023)
- [Weakly Supervised Bayesian Shape Modeling from Unsegmented Medical Images](https://mlsanity.com/paper/2405.09697/) (2024)
- [SCorP: Statistics-Informed Dense Correspondence Prediction Directly from Unsegmented Medical Images](https://mlsanity.com/paper/2404.17967/) (2024)
- [Progressive DeepSSM: Training Methodology for Image-To-Shape Deep Models](https://mlsanity.com/paper/2310.01529/) (2023)
- [MASSM: An End-to-End Deep Learning Framework for Multi-Anatomy Statistical Shape Modeling Directly From Images](https://mlsanity.com/paper/2403.11008/) (2024)
- [Mesh2SSM++: A Probabilistic Framework for Unsupervised Learning of Statistical Shape Model of Anatomies from Surface Meshes](https://mlsanity.com/paper/2502.07145/) (2025)
- [Optimization-Driven Statistical Models of Anatomies using Radial Basis Function Shape Representation](https://mlsanity.com/paper/2411.15882/) (2024)
- [Probabilistic 3D Correspondence Prediction from Sparse Unsegmented Images](https://mlsanity.com/paper/2407.01931/) (2024)
- [On the Viability of Semi-Supervised Segmentation Methods for Statistical Shape Modeling](https://mlsanity.com/paper/2407.15260/) (2024)
- [Uncertainty Quantification for Cardiac Shape Reconstruction with Deep Signed Distance Functions via MCMC methods](https://mlsanity.com/paper/2605.07987/) (2026)

## Abstract (arXiv)

Statistical shape modeling (SSM) directly from 3D medical images is an underutilized tool for detecting pathology, diagnosing disease, and conducting population-level morphology analysis. Deep learning frameworks have increased the feasibility of adopting SSM in medical practice by reducing the expert-driven manual and computational overhead in traditional SSM workflows. However, translating such frameworks to clinical practice requires calibrated uncertainty measures as neural networks can produce over-confident predictions that cannot be trusted in sensitive clinical decision-making. Existing techniques for predicting shape with aleatoric (data-dependent) uncertainty utilize a principal component analysis (PCA) based shape representation computed in isolation from the model training. This constraint restricts the learning task to solely estimating pre-defined shape descriptors from 3D images and imposes a linear relationship between this shape representation and the output (i.e., shape) space. In this paper, we propose a principled framework based on the variational information bottleneck theory to relax these assumptions while predicting probabilistic shapes of anatomy directly from images without supervised encoding of shape descriptors. Here, the latent representation is learned in the context of the learning task, resulting in a more scalable, flexible model that better captures data non-linearity. Additionally, this model is self-regularized and generalizes better given limited training data. Our experiments demonstrate that the proposed method provides improved accuracy and better calibrated aleatoric uncertainty estimates than state-of-the-art methods.