mlsanityData to 7 Oct 2026

PolyDiff: Generating 3D Polygonal Meshes with Diffusion Models

arXiv 2312.11417 ↗First posted 18 Dec 2023cs.CVcited by 66

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We introduce PolyDiff, the first diffusion-based approach capable of directly generating realistic and diverse 3D polygonal meshes. In contrast to methods that use alternate 3D shape representations (e.g. implicit representations), our approach is a discrete denoising diffusion probabilistic model that operates natively on the polygonal mesh data structure. This enables learning of both the geometric properties of vertices and the topological characteristics of faces. Specifically, we treat meshes as quantized triangle soups, progressively corrupted with categorical noise in the forward diffusion phase. In the reverse diffusion phase, a transformer-based denoising network is trained to revert the noising process, restoring the original mesh structure. At inference, new meshes can be generated by applying this denoising network iteratively, starting with a completely noisy triangle soup. Consequently, our model is capable of producing high-quality 3D polygonal meshes, ready for integration into downstream 3D workflows. Our extensive experimental analysis shows that PolyDiff achieves a significant advantage (avg. FID and JSD improvement of 18.2 and 5.8 respectively) over current state-of-the-art methods.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 27 of its 35 references that are in the map. First posted on arXiv 18 Dec 2023.

Builds on

27 of 35 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023PolyDiffuse: Polygonal Shape Reconstruction via Guided Set Diffusion Models2 Jun
  2. 2022HouseDiffusion: Vector Floorplan Generation via a Diffusion Model with Discrete and Continuous Denoising23 Nov
  3. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  4. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 23

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  9. 2023Consistency Modelscited by 1,778
  10. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  11. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  12. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
11 more
  1. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  2. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  3. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  4. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  5. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  6. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  7. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  8. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 205
  9. 2023MeshDiffusion: Score-based Generative 3D Mesh Modelingcited by 198
  10. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  11. 2020Learning Deformable Tetrahedral Meshes for 3D Reconstructioncited by 84

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 23; 2026: 19.

66 papers.

Sort:

By area:

  1. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  2. 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 140
  3. 2024MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenizationcited by 102
  4. 2024BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometrycited by 101
  5. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 100
  6. 2024Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scalecited by 87
  7. 2024Scaling Mesh Generation via Compressive Tokenizationcited by 71
  8. 2024You See it, You Got it: Learning 3D Creation on Pose-Free Videos at Scalecited by 65
  9. 2024Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objectscited by 52
  10. 2024PivotMesh: Generic 3D Mesh Generation via Pivot Vertices Guidancecited by 47
  11. 2024MeshArt: Generating Articulated Meshes with Structure-Guided Transformerscited by 40
  12. 2024DynamicCity: Large-Scale 4D Occupancy Generation from Dynamic Scenescited by 37
38 more
  1. 2024GenCAD: Image-Conditioned Computer-Aided Design Generation with Transformer-Based Contrastive Representation and Diffusion Priorscited by 36
  2. 2024An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusioncited by 34
  3. 2024PrEditor3D: Fast and Precise 3D Shape Editingcited by 27
  4. 2025Mesh-RFT: Enhancing Mesh Generation via Fine-grained Reinforcement Fine-Tuningcited by 21
  5. 2024TetSphere Splatting: Representing High-Quality Geometry with Lagrangian Volumetric Meshescited by 20
  6. 2025Nautilus: Locality-aware Autoencoder for Scalable Mesh Generationcited by 18
  7. 2025MeshLLM: Empowering Large Language Models to Progressively Understand and Generate 3D Meshcited by 18
  8. 2025LL3M: Large Language 3D Modelerscited by 18
  9. 2025FastMesh: Efficient Artistic Mesh Generation via Component Decouplingcited by 17
  10. 2024PuzzleFusion++: Auto-agglomerative 3D Fracture Assembly by Denoise and Verifycited by 14
  11. 2025MeshCraft: Exploring Efficient and Controllable Mesh Generation with Flow-based DiTscited by 14
  12. 2025VertexRegen: Mesh Generation with Continuous Level of Detailcited by 14
  13. 2024NeuSDFusion: A Spatial-Aware Generative Model for 3D Shape Completion, Reconstruction, and Generationcited by 13
  14. 2025Assembler: Scalable 3D Part Assembly via Anchor Point Diffusioncited by 11
  15. 2026LATO: 3D Mesh Flow Matching with Structured TOpology Preserving LAtentscited by 10
  16. 2024A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervisioncited by 9
  17. 2024Coherent 3D Scene Diffusion From a Single RGB Imagecited by 8
  18. 2025MeshMosaic: Scaling Artist Mesh Generation via Local-to-Global Assemblycited by 8
  19. 2024DI-PCG: Diffusion-based Efficient Inverse Procedural Content Generation for High-quality 3D Asset Creationcited by 7
  20. 2025CLR-Wire: Towards Continuous Latent Representations for 3D Curve Wireframe Generationcited by 7
  21. 2026MeshFlow: Efficient Artistic Mesh Generation via MeshVAE and Flow-based Diffusion Transformercited by 7
  22. 2025MeshPad: Interactive Sketch-Conditioned Artist-Reminiscent Mesh Generation and Editingcited by 6
  23. 2025ASIA: Adaptive 3D Segmentation using Few Image Annotationscited by 6
  24. 2026LATO.2: Factorized 3D Mesh Generation with Vertex and Topology Flowcited by 6
  25. 2025SPGen: Spherical Projection as Consistent and Flexible Representation for Single Image 3D Shape Generationcited by 5
  26. 2025Topology Sculptor, Shape Refiner: Discrete Diffusion Model for High-Fidelity 3D Meshes Generationcited by 5
  27. 2026Nexus: Native Mesh Generation with Diffusioncited by 5
  28. 2026FACE: A Face-based Autoregressive Representation for High-Fidelity and Efficient Mesh Generationcited by 4
  29. 2024GenUDC: High Quality 3D Mesh Generation with Unsigned Dual Contouring Representationcited by 3
  30. 2025Autoregressive Generation of Static and Growing Treescited by 3
  31. 2026PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstructioncited by 3
  32. 2026MeshFlow: Mesh Generation with Equivariant Flow Matchingcited by 3
  33. 2024DiffSurf: A Transformer-based Diffusion Model for Generating and Reconstructing 3D Surfaces in Posecited by 2
  34. 2026DB-3DME: From Dataset to Benchmark for Human-aligned Automatic 3D Mesh Evaluationcited by 2
  35. 2025Artist-Created Mesh Generation from Raw Observationcited by 1
  36. 2025MamTiff-CAD: Multi-Scale Latent Diffusion with Mamba+ for Complex Parametric Sequencecited by 1
  37. 2026BoxSplitGen: A Generative Model for 3D Part Bounding Boxes in Varying Granularitycited by 1
  38. 2026BuildAnyPoint: 3D Building Structured Abstraction from Diverse Point Cloudscited by 1

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022TetraDiffusion: Tetrahedral Diffusion Models for 3D Shape Generation
  2. 2025DDPM-Polycube: A Denoising Diffusion Probabilistic Model for Polycube-Based Hexahedral Mesh Generation and Volumetric Spline Construction
  3. 2026Rethinking 3D Shape Generation: Diffusion over Superquadrics
  4. 2023Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape
  5. 2022RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation
  6. 2023Single Mesh Diffusion Models with Field Latents for Texture Generation
  7. 2024DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans
  8. 2025DoubleDiffusion: Combining Heat Diffusion with Denoising Diffusion for Texture Generation on 3D Meshes