# Capturing Shape Information with Multi-Scale Topological Loss Terms for 3D Reconstruction

arXiv: https://arxiv.org/abs/2203.01703 · HTML: https://mlsanity.com/paper/2203.01703/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 14 of 14)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [clDice -- A Novel Topology-Preserving Loss Function for Tubular Structure Segmentation](https://mlsanity.com/paper/2003.07311/) (2020)
- [Topology-Preserving Deep Image Segmentation](https://mlsanity.com/paper/1906.05404/) (2019)
- [A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology](https://mlsanity.com/paper/1910.01877/) (2019)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Topology-Aware Segmentation Using Discrete Morse Theory](https://mlsanity.com/paper/2103.09992/) (2021)
- [Graph Filtration Learning](https://mlsanity.com/paper/1905.10996/) (2019)
- [Uncovering the Topology of Time-Varying fMRI Data using Cubical Persistence](https://mlsanity.com/paper/2006.07882/) (2020)
- [Interactive Visualization of 2-D Persistence Modules](https://mlsanity.com/paper/1512.00180/) (2015)
- [The Impact of Changes in Resolution on the Persistent Homology of Images](https://mlsanity.com/paper/2111.05663/) (2021)

## Built on by (most cited first, 14 of 14)

- [Position: Topological Deep Learning is the New Frontier for Relational Learning](https://mlsanity.com/paper/2402.08871/) (2024)
- [Topologically faithful image segmentation via induced matching of persistence barcodes](https://mlsanity.com/paper/2211.15272/) (2022)
- [A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images](https://mlsanity.com/paper/2208.14125/) (2022)
- [Euler Characteristic Transform Based Topological Loss for Reconstructing 3D Images from Single 2D Slices](https://mlsanity.com/paper/2303.05286/) (2023)
- [Self Pre-training with Topology- and Spatiality-aware Masked Autoencoders for 3D Medical Image Segmentation](https://mlsanity.com/paper/2406.10519/) (2024)
- [Mapping the Multiverse of Latent Representations](https://mlsanity.com/paper/2402.01514/) (2024)
- [Topology meets Machine Learning: An Introduction using the Euler Characteristic Transform](https://mlsanity.com/paper/2410.17760/) (2024)
- [Towards Scalable Topological Regularizers](https://mlsanity.com/paper/2501.14641/) (2025)
- [Molecular Machine Learning Using Euler Characteristic Transforms](https://mlsanity.com/paper/2507.03474/) (2025)
- [Topological Inductive Bias fosters Multiple Instance Learning in Data-Scarce Scenarios](https://mlsanity.com/paper/2307.14025/) (2023)
- [Scalar Function Topology Divergence: Comparing Topology of 3D Objects](https://mlsanity.com/paper/2407.08364/) (2024)
- [SDF-TopoNet: A Two-Stage Framework for Tubular Structure Segmentation via SDF Pre-training and Topology-Aware Fine-Tuning](https://mlsanity.com/paper/2503.14523/) (2025)
- [Reconstruction of three-dimensional shapes of normal and disease-related erythrocytes from partial observations using multi-fidelity neural networks](https://mlsanity.com/paper/2511.14962/) (2025)
- [TopoEmbedX: A General Framework for Representation Learning on Topological Domains](https://mlsanity.com/paper/2609.37884/) (2026)

## Abstract (arXiv)

Reconstructing 3D objects from 2D images is both challenging for our brains and machine learning algorithms. To support this spatial reasoning task, contextual information about the overall shape of an object is critical. However, such information is not captured by established loss terms (e.g. Dice loss). We propose to complement geometrical shape information by including multi-scale topological features, such as connected components, cycles, and voids, in the reconstruction loss. Our method uses cubical complexes to calculate topological features of 3D volume data and employs an optimal transport distance to guide the reconstruction process. This topology-aware loss is fully differentiable, computationally efficient, and can be added to any neural network. We demonstrate the utility of our loss by incorporating it into SHAPR, a model for predicting the 3D cell shape of individual cells based on 2D microscopy images. Using a hybrid loss that leverages both geometrical and topological information of single objects to assess their shape, we find that topological information substantially improves the quality of reconstructions, thus highlighting its ability to extract more relevant features from image datasets.