mlsanityData to 7 Oct 2026

MatAtlas: Text-driven Consistent Geometry Texturing and Material Assignment

arXiv 2404.02899 ↗First posted 3 Apr 2024cs.CV cs.GRcited by 14

Not on the map. Why?

Abstract (arXiv)

We present MatAtlas, a method for consistent text-guided 3D model texturing. Following recent progress we leverage a large scale text-to-image generation model (e.g., Stable Diffusion) as a prior to texture a 3D model. We carefully design an RGB texturing pipeline that leverages a grid pattern diffusion, driven by depth and edges. By proposing a multi-step texture refinement process, we significantly improve the quality and 3D consistency of the texturing output. To further address the problem of baked-in lighting, we move beyond RGB colors and pursue assigning parametric materials to the assets. Given the high-quality initial RGB texture, we propose a novel material retrieval method capitalized on Large Language Models (LLM), enabling editabiliy and relightability. We evaluate our method on a wide variety of geometries and show that our method significantly outperform prior arts. We also analyze the role of each component through a detailed ablation study.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 44 of its 77 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

44 of 77 references.

References · 44

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  8. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  9. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  10. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  12. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
32 more
  1. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  2. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  3. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  4. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  5. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  6. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  7. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
  8. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  9. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  10. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  11. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  12. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  13. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  14. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  15. 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
  16. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 234
  17. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  18. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Priorcited by 157
  19. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  20. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3Dcited by 134
  21. 2023TextDeformer: Geometry Manipulation using Text Guidancecited by 94
  22. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 86
  23. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  24. 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
  25. 2023Texture Generation on 3D Meshes with Point-UV Diffusioncited by 63
  26. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  27. 2024TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusioncited by 51
  28. 2020Guided Fine-Tuning for Large-Scale Material Transfercited by 27
  29. 2023PSDR-Room: Single Photo to Scene using Differentiable Renderingcited by 23
  30. 2023Breathing New Life into 3D Assets with Generative Repaintingcited by 17
  31. 2023Text-guided High-definition Consistency Texture Modelcited by 7
  32. 2023The Visual Language of Fabricscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 8; 2026: 1.

14 papers.

Sort:

By area:

  1. 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 120
  2. 2024TEXGen: a Generative Diffusion Model for Mesh Texturescited by 46
  3. 2024Instant3dit: Multiview Inpainting for Fast Editing of 3D Objectscited by 43
  4. 2025RomanTex: Decoupling 3D-aware Rotary Positional Embedded Multi-Attention Network for Texture Synthesiscited by 20
  5. 2025VideoMat: Extracting PBR Materials from Video Diffusion Modelscited by 10
  6. 2024SAMa: Material-aware 3D Selection and Segmentationcited by 9
  7. 2025MARBLE: Material Recomposition and Blending in CLIP-Spacecited by 9
  8. 20253D Stylization via Large Reconstruction Modelcited by 8
  9. 2024MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assetscited by 6
  10. 2025Im2SurfTex: Surface Texture Generation via Neural Backprojection of Multi-View Imagescited by 3
  11. 2025MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Modelscited by 2
  12. 2025PacTure: Efficient PBR Texture Generation on Packed Views with Visual Autoregressive Modelscited by 2
2 more
  1. 2024ConTEXTure: Consistent Multiview Images to Texturecited by 0
  2. 2026Texture Space Material Diffusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MaPa: Text-driven Photorealistic Material Painting for 3D Shapes
  2. 2024GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation
  3. 2023Text-Guided Texturing by Synchronized Multi-View Diffusion
  4. 2025MD-ProjTex: Texturing 3D Shapes with Multi-Diffusion Projection
  5. 2024RoCoTex: A Robust Method for Consistent Texture Synthesis with Diffusion Models
  6. 2025MuMA: 3D PBR Texturing via Multi-Channel Multi-View Generation and Agentic Post-Processing
  7. 2024An Optimization Framework to Enforce Multi-View Consistency for Texturing 3D Meshes
  8. 2024Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials