mlsanityData to 14 Oct 2026

Single Mesh Diffusion Models with Field Latents for Texture Generation

arXiv 2312.09250 ↗First posted 14 Dec 2023cs.CV cs.GR cs.LGcited by 11

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We introduce a framework for intrinsic latent diffusion models operating directly on the surfaces of 3D shapes, with the goal of synthesizing high-quality textures. Our approach is underpinned by two contributions: field latents, a latent representation encoding textures as discrete vector fields on the mesh vertices, and field latent diffusion models, which learn to denoise a diffusion process in the learned latent space on the surface. We consider a single-textured-mesh paradigm, where our models are trained to generate variations of a given texture on a mesh. We show the synthesized textures are of superior fidelity compared those from existing single-textured-mesh generative models. Our models can also be adapted for user-controlled editing tasks such as inpainting and label-guided generation. The efficacy of our approach is due in part to the equivariance of our proposed framework under isometries, allowing our models to seamlessly reproduce details across locally similar regions and opening the door to a notion of generative texture transfer.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 49 of its 65 references that are in the map. First posted on arXiv 14 Dec 2023.

Builds on

49 of 65 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models20 Oct
  2. 2023Breathing New Life into 3D Assets with Generative Repainting15 Sep
  3. 2023Texture Generation on 3D Meshes with Point-UV Diffusion21 Aug
  4. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  5. 2023Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shape24 May
  6. 2023Text-guided High-definition Consistency Texture Model10 May
  7. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Models20 Mar
  8. 2023DINAR: Diffusion Inpainting of Neural Textures for One-Shot Human Avatars16 Mar
  9. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  10. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 39

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 657
  7. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  8. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  9. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  10. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  11. 2021Vector Neurons: A General Framework for SO(3)-Equivariant Networkscited by 305
  12. 2020Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Datacited by 248
27 more
  1. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  2. 2023MeshDiffusion: Score-based Generative 3D Mesh Modelingcited by 199
  3. 2020DiffusionNet: Discretization Agnostic Learning on Surfacescited by 164
  4. 2020Improved Techniques for Training Single-Image GANscited by 78
  5. 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
  6. 2021Subdivision-Based Mesh Convolution Networkscited by 60
  7. 2022SinDDM: A Single Image Denoising Diffusion Modelcited by 60
  8. 2022SinFusion: Training Diffusion Models on a Single Image or Videocited by 58
  9. 2021Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Modelscited by 51
  10. 2019Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networkscited by 45
  11. 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
  12. 2020CNNs on Surfaces using Rotation-Equivariant Featurescited by 43
  13. 2022SinDiffusion: Learning a Diffusion Model from a Single Natural Imagecited by 42
  14. 2022AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesiscited by 39
  15. 2023Mesh2Tex: Generating Mesh Textures from Image Queriescited by 31
  16. 2023Diffusion Probabilistic Fieldscited by 28
  17. 2023TAPS3D: Text-Guided 3D Textured Shape Generation from Pseudo Supervisioncited by 28
  18. 2022VN-Transformer: Rotation-Equivariant Attention for Vector Neuronscited by 27
  19. 2023GETAvatar: Generative Textured Meshes for Animatable Human Avatarscited by 27
  20. 2022Intrinsic Neural Fields: Learning Functions on Manifoldscited by 20
  21. 2021Complex Functional Maps : a Conformal Link Between Tangent Bundlescited by 17
  22. 2021Field Convolutions for Surface CNNscited by 14
  23. 2022M"obius Convolutions for Spherical CNNscited by 13
  24. 2022Rotation-Equivariant Conditional Spherical Neural Fields for Learning a Natural Illumination Priorcited by 12
  25. 2023Manifold Diffusion Fieldscited by 11
  26. 2023TUVF: Learning Generalizable Texture UV Radiance Fieldscited by 9
  27. 2020Efficient Spatially Adaptive Convolution and Correlationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 4; 2026: 3.

11 papers.

  1. 2024FlexiTex: Enhancing Texture Generation via Visual Guidancecited by 8
  2. 2024UV-free Texture Generation with Denoising and Geodesic Heat Diffusionscited by 3
  3. 2025ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusioncited by 3
  4. 2025GT2-GS: Geometry-aware Texture Transfer for Gaussian Splattingcited by 3
  5. 2025TriTex: Learning Texture from a Single Mesh via Triplane Semantic Featurescited by 2
  6. 2025DoubleDiffusion: Combining Heat Diffusion with Denoising Diffusion for Texture Generation on 3D Meshescited by 1
  7. 2024CFCPalsy: Facial Image Synthesis with Cross-Fusion Cycle Diffusion Model for Facial Paralysis Individualscited by 0
  8. 2024Mat'ern Kernels for Tunable Implicit Surface Reconstructioncited by 0
  9. 2026Smoothness Errors in Dynamics Models and How to Avoid Themcited by 0
  10. 2026Advances in Neural 3D Mesh Texturing: A Surveycited by 0
  11. 2026GLOSS: Geometric Local Self-Similarity Learning for Faithful Reference-Guided Texture Fillcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generation
  2. 2023Diffusion-based Holistic Texture Rectification and Synthesis
  3. 2025MatLat: Material Latent Space for PBR Texture Generation
  4. 2026Structured 3D Latents Are Surprisingly Powerful: Unleashing Generalizable Style with 2D Diffusion
  5. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation
  6. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior
  7. 2024GenesisTex: Adapting Image Denoising Diffusion to Texture Space
  8. 2025Semantics Lead the Way: Harmonizing Semantic and Texture Modeling with Asynchronous Latent Diffusion