mlsanityData to 7 Oct 2026

Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis

arXiv 2111.04276 ↗First posted 8 Nov 2021cs.CV cs.LGcited by 459

Not on the map. Why?

Cited by 131 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce DMTet, a deep 3D conditional generative model that can synthesize high-resolution 3D shapes using simple user guides such as coarse voxels. It marries the merits of implicit and explicit 3D representations by leveraging a novel hybrid 3D representation. Compared to the current implicit approaches, which are trained to regress the signed distance values, DMTet directly optimizes for the reconstructed surface, which enables us to synthesize finer geometric details with fewer artifacts. Unlike deep 3D generative models that directly generate explicit representations such as meshes, our model can synthesize shapes with arbitrary topology. The core of DMTet includes a deformable tetrahedral grid that encodes a discretized signed distance function and a differentiable marching tetrahedra layer that converts the implicit signed distance representation to the explicit surface mesh representation. This combination allows joint optimization of the surface geometry and topology as well as generation of the hierarchy of subdivisions using reconstruction and adversarial losses defined explicitly on the surface mesh. Our approach significantly outperforms existing work on conditional shape synthesis from coarse voxel inputs, trained on a dataset of complex 3D animal shapes. Project page: https://nv-tlabs.github.io/DMTet/.

Key facts. Cited by 459 papers in the map (data through 7 Oct 2026). Builds on 48 of its 73 references that are in the map. First posted on arXiv 8 Nov 2021.

Builds on

48 of 73 references.

References · 48

  1. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  6. 2016Least Squares Generative Adversarial Networkscited by 1,000
  7. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  8. 2020Convolutional Occupancy Networkscited by 760
  9. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  10. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  11. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  12. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
36 more
  1. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  2. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  3. 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
  4. 2019Point-Voxel CNN for Efficient 3D Deep Learningcited by 343
  5. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  6. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  7. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  8. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  9. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  10. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  11. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 205
  12. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  13. 2016Learning Shape Abstractions by Assembling Volumetric Primitivescited by 181
  14. 2016Generative and Discriminative Voxel Modeling with Convolutional Neural Networkscited by 128
  15. 2020Deformed Implicit Field: Modeling 3D Shapes with Learned Dense Correspondencecited by 119
  16. 2019SDM-NET: Deep Generative Network for Structured Deformable Meshcited by 113
  17. 2020MeshSDF: Differentiable Iso-Surface Extractioncited by 112
  18. 2017ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scanscited by 103
  19. 2021Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networkscited by 101
  20. 2019Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Researchcited by 98
  21. 2019Fast Interactive Object Annotation with Curve-GCNcited by 95
  22. 2017Hierarchical Surface Prediction for 3D Object Reconstructioncited by 89
  23. 2020Learning Deformable Tetrahedral Meshes for 3D Reconstructioncited by 84
  24. 2020DualSDF: Semantic Shape Manipulation using a Two-Level Representationcited by 82
  25. 2020Curriculum DeepSDFcited by 62
  26. 2020Neural Splines: Fitting 3D Surfaces with Infinitely-Wide Neural Networkscited by 52
  27. 2020Neural Subdivisioncited by 49
  28. 2020Adversarial Generation of Continuous Implicit Shape Representationscited by 46
  29. 2020DECOR-GAN: 3D Shape Detailization by Conditional Refinementcited by 45
  30. 2020D2IM-Net: Learning Detail Disentangled Implicit Fields from Single Imagescited by 42
  31. 2020Deep Octree-based CNNs with Output-Guided Skip Connections for 3D Shape and Scene Completioncited by 33
  32. 2020COALESCE: Component Assembly by Learning to Synthesize Connectionscited by 30
  33. 2020On the Effectiveness of Weight-Encoded Neural Implicit 3D Shapescited by 29
  34. 2020Analytic Marching: An Analytic Meshing Solution from Deep Implicit Surface Networkscited by 16
  35. 2020Beyond Fixed Grid: Learning Geometric Image Representation with a Deformable Gridcited by 16
  36. 2019DeepSpline: Data-Driven Reconstruction of Parametric Curves and Surfacescited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 19; 2023: 131; 2024: 183; 2025: 91; 2026: 32.

459 papers.

Sort:

By area:

  1. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  2. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  3. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  4. 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 730
  5. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  6. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  7. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  8. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  9. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  10. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  11. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  12. 2023Shap-E: Generating Conditional 3D Implicit Functionscited by 379
38 more
  1. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  2. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  3. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 308
  4. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  5. 2023Text-to-3D using Gaussian Splattingcited by 299
  6. 2023LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matchingcited by 290
  7. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  8. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  9. 2023MeshGPT: Generating Triangle Meshes with Decoder-Only Transformerscited by 254
  10. 2024Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenescited by 226
  11. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 225
  12. 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
  13. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  14. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  15. 2023MeshDiffusion: Score-based Generative 3D Mesh Modelingcited by 198
  16. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  17. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  18. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  19. 2023NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Imagescited by 163
  20. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  21. 2023DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Modelscited by 154
  22. 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 140
  23. 2023Flexible Isosurface Extraction for Gradient-Based Mesh Optimizationcited by 138
  24. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  25. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3Dcited by 134
  26. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 132
  27. 2024Recent Advances in 3D Gaussian Splattingcited by 130
  28. 2023Text-to-3D with Classifier Score Distillationcited by 127
  29. 20234DGen: Grounded 4D Content Generation with Spatial-temporal Consistencycited by 117
  30. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  31. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
  32. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidancecited by 109
  33. 2023TADA! Text to Animatable Digital Avatarscited by 109
  34. 2023Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinementcited by 106
  35. 2023TeCH: Text-guided Reconstruction of Lifelike Clothed Humanscited by 104
  36. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 103
  37. 2023Neural Kernel Surface Reconstructioncited by 102
  38. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102

50 most cited of 459 All 459 (JSON)