# Learning Deformable Tetrahedral Meshes for 3D Reconstruction

arXiv: https://arxiv.org/abs/2011.01437 · HTML: https://mlsanity.com/paper/2011.01437/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2020.

## Built on by (most cited first, 30 of 85)

- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising](https://mlsanity.com/paper/2206.03380/) (2022)
- [MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers](https://mlsanity.com/paper/2406.10163/) (2024)
- [Flexible Isosurface Extraction for Gradient-Based Mesh Optimization](https://mlsanity.com/paper/2308.05371/) (2023)
- [PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objects](https://mlsanity.com/paper/2308.07391/) (2023)
- [Neural Kernel Surface Reconstruction](https://mlsanity.com/paper/2305.19590/) (2023)
- [TeCH: Text-guided Reconstruction of Lifelike Clothed Humans](https://mlsanity.com/paper/2308.08545/) (2023)
- [Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks](https://mlsanity.com/paper/2103.10429/) (2021)
- [gradSim: Differentiable simulation for system identification and visuomotor control](https://mlsanity.com/paper/2104.02646/) (2021)
- [POCO: Point Convolution for Surface Reconstruction](https://mlsanity.com/paper/2201.01831/) (2022)
- [Neural Dual Contouring](https://mlsanity.com/paper/2202.01999/) (2022)
- [Adaptive Shells for Efficient Neural Radiance Field Rendering](https://mlsanity.com/paper/2311.10091/) (2023)
- [PolyDiff: Generating 3D Polygonal Meshes with Diffusion Models](https://mlsanity.com/paper/2312.11417/) (2023)
- [Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materials](https://mlsanity.com/paper/2407.02445/) (2024)
- [Texture Generation on 3D Meshes with Point-UV Diffusion](https://mlsanity.com/paper/2308.10490/) (2023)
- [GAvatar: Animatable 3D Gaussian Avatars with Implicit Mesh Learning](https://mlsanity.com/paper/2312.11461/) (2023)
- [3DStyleNet: Creating 3D Shapes with Geometric and Texture Style Variations](https://mlsanity.com/paper/2108.12958/) (2021)
- [Neural Fields as Learnable Kernels for 3D Reconstruction](https://mlsanity.com/paper/2111.13674/) (2021)
- [Urban Radiance Field Representation with Deformable Neural Mesh Primitives](https://mlsanity.com/paper/2307.10776/) (2023)
- [DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer](https://mlsanity.com/paper/2111.00140/) (2021)
- [GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering](https://mlsanity.com/paper/2403.11324/) (2024)
- [Neural Marching Cubes](https://mlsanity.com/paper/2106.11272/) (2021)
- [HeadSculpt: Crafting 3D Head Avatars with Text](https://mlsanity.com/paper/2306.03038/) (2023)
- [Surface Reconstruction from Point Clouds: A Survey and a Benchmark](https://mlsanity.com/paper/2205.02413/) (2022)
- [Controllable Mesh Generation Through Sparse Latent Point Diffusion Models](https://mlsanity.com/paper/2303.07938/) (2023)

## Abstract (arXiv)

3D shape representations that accommodate learning-based 3D reconstruction are an open problem in machine learning and computer graphics. Previous work on neural 3D reconstruction demonstrated benefits, but also limitations, of point cloud, voxel, surface mesh, and implicit function representations. We introduce Deformable Tetrahedral Meshes (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem. Unlike existing volumetric approaches, DefTet optimizes for both vertex placement and occupancy, and is differentiable with respect to standard 3D reconstruction loss functions. It is thus simultaneously high-precision, volumetric, and amenable to learning-based neural architectures. We show that it can represent arbitrary, complex topology, is both memory and computationally efficient, and can produce high-fidelity reconstructions with a significantly smaller grid size than alternative volumetric approaches. The predicted surfaces are also inherently defined as tetrahedral meshes, thus do not require post-processing. We demonstrate that DefTet matches or exceeds both the quality of the previous best approaches and the performance of the fastest ones. Our approach obtains high-quality tetrahedral meshes computed directly from noisy point clouds, and is the first to showcase high-quality 3D tet-mesh results using only a single image as input. Our project webpage: https://nv-tlabs.github.io/DefTet/