# A Theory of Topological Derivatives for Inverse Rendering of Geometry

arXiv: https://arxiv.org/abs/2308.09865 · HTML: https://mlsanity.com/paper/2308.09865/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 54 references that are in the map.
- First posted on arXiv 19 Aug 2023.

## Builds on (most cited first, 15 of 15)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [DIST: Rendering Deep Implicit Signed Distance Function with Differentiable Sphere Tracing](https://mlsanity.com/paper/1911.13225/) (2019)
- [SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization](https://mlsanity.com/paper/1912.07109/) (2019)
- [MeshSDF: Differentiable Iso-Surface Extraction](https://mlsanity.com/paper/2006.03997/) (2020)
- [A Level Set Theory for Neural Implicit Evolution under Explicit Flows](https://mlsanity.com/paper/2204.07159/) (2022)
- [Differentiable Rendering of Neural SDFs through Reparameterization](https://mlsanity.com/paper/2206.05344/) (2022)

## Built on by (most cited first, 6 of 6)

- [Radiance Surfaces: Optimizing Surface Representations with a 5D Radiance Field Loss](https://mlsanity.com/paper/2501.18627/) (2025)
- [DiffSoup: Direct Differentiable Rasterization of Triangle Soup for Extreme Radiance Field Simplification](https://mlsanity.com/paper/2603.27151/) (2026)
- [Many-Worlds Inverse Rendering](https://mlsanity.com/paper/2408.16005/) (2024)
- [Inverse Rendering for High-Genus Surface Meshes from Multi-View Images](https://mlsanity.com/paper/2511.18680/) (2025)
- [Locally Orderless Images for Optimization in Differentiable Rendering](https://mlsanity.com/paper/2503.21931/) (2025)
- [DiffTrans: Differentiable Geometry-Materials Decomposition for Reconstructing Transparent Objects](https://mlsanity.com/paper/2603.00413/) (2026)

## Abstract (arXiv)

We introduce a theoretical framework for differentiable surface evolution that allows discrete topology changes through the use of topological derivatives for variational optimization of image functionals. While prior methods for inverse rendering of geometry rely on silhouette gradients for topology changes, such signals are sparse. In contrast, our theory derives topological derivatives that relate the introduction of vanishing holes and phases to changes in image intensity. As a result, we enable differentiable shape perturbations in the form of hole or phase nucleation. We validate the proposed theory with optimization of closed curves in 2D and surfaces in 3D to lend insights into limitations of current methods and enable improved applications such as image vectorization, vector-graphics generation from text prompts, single-image reconstruction of shape ambigrams and multi-view 3D reconstruction.