mlsanityData to 7 Oct 2026

Consistent Mesh Diffusion

arXiv 2312.00971 ↗First posted 1 Dec 2023cs.CV cs.GRcited by 3

Not on the map. Why?

Abstract (arXiv)

Given a 3D mesh with a UV parameterization, we introduce a novel approach to generating textures from text prompts. While prior work uses optimization from Text-to-Image Diffusion models to generate textures and geometry, this is slow and requires significant compute resources. Alternatively, there are projection based approaches that use the same Text-to-Image models that paint images onto a mesh, but lack consistency at different viewing angles, we propose a method that uses a single Depth-to-Image diffusion network, and generates a single consistent texture when rendered on the 3D surface by first unifying multiple 2D image's diffusion paths, and hoisting that to 3D with MultiDiffusion~citemultidiffusion. We demonstrate our approach on a dataset containing 30 meshes, taking approximately 5 minutes per mesh. To evaluate the quality of our approach, we use CLIP-score~citeclipscore and Frechet Inception Distance (FID)~citefrechet to evaluate the quality of the rendering, and show our improvement over prior work.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 56 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

34 of 56 references.

References · 34

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  8. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  9. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  11. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  12. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
22 more
  1. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  2. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  3. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  4. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  5. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  6. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  7. 2022Blended Latent Diffusioncited by 430
  8. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  9. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 234
  10. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  11. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generationcited by 181
  12. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  13. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  14. 2023Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generationcited by 133
  15. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  16. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 86
  17. 2023Texture Generation on 3D Meshes with Point-UV Diffusioncited by 63
  18. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  19. 2021Learning Generative Models of Textured 3D Meshes from Real-World Imagescited by 44
  20. 2021Appearance-Driven Automatic 3D Model Simplificationcited by 24
  21. 2022Fine Detailed Texture Learning for 3D Meshes with Generative Modelscited by 15
  22. 2023Text-guided High-definition Consistency Texture Modelcited by 7

Built on by

Cited by year

Citing papers per year: 2024: 3.

3 papers.

  1. 2024Collaborative Control for Geometry-Conditioned PBR Image Generationcited by 22
  2. 2024Jointly Generating Multi-view Consistent PBR Textures using Collaborative Controlcited by 4
  3. 2024StyleCity: Large-Scale 3D Urban Scenes Stylizationcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2024Tex4D: Zero-shot 4D Scene Texturing with Video Diffusion Models
  2. 2024An Optimization Framework to Enforce Multi-View Consistency for Texturing 3D Meshes
  3. 2023EucliDreamer: Fast and High-Quality Texturing for 3D Models with Stable Diffusion Depth
  4. 2024RoCoTex: A Robust Method for Consistent Texture Synthesis with Diffusion Models
  5. 2025Progressive Rendering Distillation: Adapting Stable Diffusion for Instant Text-to-Mesh Generation without 3D Data
  6. 2023Text-Guided Texturing by Synchronized Multi-View Diffusion
  7. 2024TexPainter: Generative Mesh Texturing with Multi-view Consistency
  8. 2024CT4D: Consistent Text-to-4D Generation with Animatable Meshes