# Single Mesh Diffusion Models with Field Latents for Texture Generation

arXiv: https://arxiv.org/abs/2312.09250 · HTML: https://mlsanity.com/paper/2312.09250/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 49 of its 65 references that are in the map.
- First posted on arXiv 14 Dec 2023.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 49)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items](https://mlsanity.com/paper/2204.11918/) (2022)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)
- [SinGAN: Learning a Generative Model from a Single Natural Image](https://mlsanity.com/paper/1905.01164/) (2019)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data](https://mlsanity.com/paper/2002.12880/) (2020)
- [Text2Tex: Text-driven Texture Synthesis via Diffusion Models](https://mlsanity.com/paper/2303.11396/) (2023)
- [Texture Fields: Learning Texture Representations in Function Space](https://mlsanity.com/paper/1905.07259/) (2019)
- [MeshDiffusion: Score-based Generative 3D Mesh Modeling](https://mlsanity.com/paper/2303.08133/) (2023)
- [DiffusionNet: Discretization Agnostic Learning on Surfaces](https://mlsanity.com/paper/2012.00888/) (2020)
- [TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models](https://mlsanity.com/paper/2310.13772/) (2023)
- [Improved Techniques for Training Single-Image GANs](https://mlsanity.com/paper/2003.11512/) (2020)
- [Texturify: Generating Textures on 3D Shape Surfaces](https://mlsanity.com/paper/2204.02411/) (2022)
- [Texture Generation on 3D Meshes with Point-UV Diffusion](https://mlsanity.com/paper/2308.10490/) (2023)
- [Subdivision-Based Mesh Convolution Networks](https://mlsanity.com/paper/2106.02285/) (2021)
- [SinDDM: A Single Image Denoising Diffusion Model](https://mlsanity.com/paper/2211.16582/) (2022)
- [SinFusion: Training Diffusion Models on a Single Image or Video](https://mlsanity.com/paper/2211.11743/) (2022)
- [Drop the GAN: In Defense of Patches Nearest Neighbors as Single Image Generative Models](https://mlsanity.com/paper/2103.15545/) (2021)
- [Filter Response Normalization Layer: Eliminating Batch Dependence in the Training of Deep Neural Networks](https://mlsanity.com/paper/1911.09737/) (2019)
- [Learning Generative Models of Textured 3D Meshes from Real-World Images](https://mlsanity.com/paper/2103.15627/) (2021)
- [CNNs on Surfaces using Rotation-Equivariant Features](https://mlsanity.com/paper/2006.01570/) (2020)

## Built on by (most cited first, 11 of 11)

- [FlexiTex: Enhancing Texture Generation via Visual Guidance](https://mlsanity.com/paper/2409.12431/) (2024)
- [UV-free Texture Generation with Denoising and Geodesic Heat Diffusions](https://mlsanity.com/paper/2408.16762/) (2024)
- [ShapeShifter: 3D Variations Using Multiscale and Sparse Point-Voxel Diffusion](https://mlsanity.com/paper/2502.02187/) (2025)
- [GT2-GS: Geometry-aware Texture Transfer for Gaussian Splatting](https://mlsanity.com/paper/2505.15208/) (2025)
- [TriTex: Learning Texture from a Single Mesh via Triplane Semantic Features](https://mlsanity.com/paper/2503.16630/) (2025)
- [DoubleDiffusion: Combining Heat Diffusion with Denoising Diffusion for Texture Generation on 3D Meshes](https://mlsanity.com/paper/2501.03397/) (2025)
- [CFCPalsy: Facial Image Synthesis with Cross-Fusion Cycle Diffusion Model for Facial Paralysis Individuals](https://mlsanity.com/paper/2409.07271/) (2024)
- [Mat'ern Kernels for Tunable Implicit Surface Reconstruction](https://mlsanity.com/paper/2409.15466/) (2024)
- [Smoothness Errors in Dynamics Models and How to Avoid Them](https://mlsanity.com/paper/2602.05352/) (2026)
- [Advances in Neural 3D Mesh Texturing: A Survey](https://mlsanity.com/paper/2606.00137/) (2026)
- [GLOSS: Geometric Local Self-Similarity Learning for Faithful Reference-Guided Texture Fill](https://mlsanity.com/paper/2608.25461/) (2026)

## Abstract (arXiv)

We introduce a framework for intrinsic latent diffusion models operating directly on the surfaces of 3D shapes, with the goal of synthesizing high-quality textures. Our approach is underpinned by two contributions: field latents, a latent representation encoding textures as discrete vector fields on the mesh vertices, and field latent diffusion models, which learn to denoise a diffusion process in the learned latent space on the surface. We consider a single-textured-mesh paradigm, where our models are trained to generate variations of a given texture on a mesh. We show the synthesized textures are of superior fidelity compared those from existing single-textured-mesh generative models. Our models can also be adapted for user-controlled editing tasks such as inpainting and label-guided generation. The efficacy of our approach is due in part to the equivariance of our proposed framework under isometries, allowing our models to seamlessly reproduce details across locally similar regions and opening the door to a notion of generative texture transfer.