mlsanityData to 7 Oct 2026

Text2Tex: Text-driven Texture Synthesis via Diffusion Models

arXiv 2303.11396 ↗First posted 20 Mar 2023cs.CVcited by 234

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present Text2Tex, a novel method for generating high-quality textures for 3D meshes from the given text prompts. Our method incorporates inpainting into a pre-trained depth-aware image diffusion model to progressively synthesize high resolution partial textures from multiple viewpoints. To avoid accumulating inconsistent and stretched artifacts across views, we dynamically segment the rendered view into a generation mask, which represents the generation status of each visible texel. This partitioned view representation guides the depth-aware inpainting model to generate and update partial textures for the corresponding regions. Furthermore, we propose an automatic view sequence generation scheme to determine the next best view for updating the partial texture. Extensive experiments demonstrate that our method significantly outperforms the existing text-driven approaches and GAN-based methods.

Key facts. Cited by 234 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 61 of its 71 references that are in the map. First posted on arXiv 20 Mar 2023.

Builds on

61 of 71 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  2. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
  3. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  4. 2022LION: Latent Point Diffusion Models for 3D Shape Generation12 Oct
  5. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep
  6. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan
  7. 2021Diffusion Probabilistic Models for 3D Point Cloud Generation2 Mar

Other references · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  12. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
38 more
  1. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  2. 2018Demystifying MMD GANscited by 1,268
  3. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  4. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  5. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  6. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  7. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  8. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  9. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  10. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  11. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  12. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  13. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 646
  14. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  15. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  16. 2019ScanRefer: 3D Object Localization in RGB-D Scans using Natural Languagecited by 446
  17. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  18. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  19. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  20. 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 287
  21. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  22. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  23. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  24. 2020Scan2Cap: Context-aware Dense Captioning in RGB-D Scanscited by 215
  25. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  26. 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
  27. 2021UniT: Multimodal Multitask Learning with a Unified Transformercited by 195
  28. 2015A Large-Scale Car Dataset for Fine-Grained Categorization and Verificationcited by 129
  29. 2022UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Groundingcited by 69
  30. 2018Learning Shape Priors for Single-View 3D Completion and Reconstructioncited by 68
  31. 2023InfiniCity: Infinite-Scale City Synthesiscited by 68
  32. 2017Improved Adversarial Systems for 3D Object Generation and Reconstructioncited by 66
  33. 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
  34. 2022DisCoScene: Spatially Disentangled Generative Radiance Fields for Controllable 3D-aware Scene Synthesiscited by 61
  35. 2018Learning Descriptor Networks for 3D Shape Synthesis and Analysiscited by 56
  36. 2021Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketchescited by 54
  37. 2021D3Net: A Unified Speaker-Listener Architecture for 3D Dense Captioning and Visual Groundingcited by 52
  38. 20233D generation on ImageNetcited by 47

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 51; 2024: 97; 2025: 60; 2026: 26.

234 papers.

Sort:

By area:

  1. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  2. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  3. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  4. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 444
  5. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  6. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
  7. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
  8. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  9. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Modelscited by 122
  10. 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 120
  11. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 111
  12. 2023SyncDiffusion: Coherent Montage via Synchronized Joint Diffusionscited by 108
38 more
  1. 2024GES: Generalized Exponential Splatting for Efficient Radiance Field Renderingcited by 92
  2. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Modelscited by 86
  3. 2023Text-Guided Texturing by Synchronized Multi-View Diffusioncited by 86
  4. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 85
  5. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  6. 2023AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homescited by 81
  7. 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 81
  8. 2024EchoScene: Indoor Scene Generation via Information Echo over Scene Graph Diffusioncited by 66
  9. 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
  10. 2024Advances in 3D Generation: A Surveycited by 62
  11. 2023Generative AI meets 3D: A Survey on Text-to-3D in AIGC Eracited by 61
  12. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  13. 2023Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Featurescited by 55
  14. 2024Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objectscited by 52
  15. 2024TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusioncited by 51
  16. 2023Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusioncited by 50
  17. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
  18. 2023HeadSculpt: Crafting 3D Head Avatars with Textcited by 48
  19. 2024Generic 3D Diffusion Adapter Using Controlled Multi-View Editingcited by 46
  20. 2024TEXGen: a Generative Diffusion Model for Mesh Texturescited by 46
  21. 2023SceneTex: High-Quality Texture Synthesis for Indoor Scenes via Diffusion Priorscited by 43
  22. 2024DressCode: Autoregressively Sewing and Generating Garments from Text Guidancecited by 43
  23. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 41
  24. 2023Point-Cloud Completion with Pretrained Text-to-image Diffusion Modelscited by 37
  25. 2023Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generationcited by 37
  26. 2024Rethinking Score Distillation as a Bridge Between Image Distributionscited by 36
  27. 20233D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillationcited by 34
  28. 2024MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3Dcited by 34
  29. 2023Disentangled Clothed Avatar Generation from Text Descriptionscited by 33
  30. 2023Sherpa3D: Boosting High-Fidelity Text-to-3D Generation via Coarse 3D Priorcited by 32
  31. 2024View Selection for 3D Captioning via Diffusion Rankingcited by 31
  32. 2024Material Anything: Generating Materials for Any 3D Object via Diffusioncited by 31
  33. 2024TexPainter: Generative Mesh Texturing with Multi-view Consistencycited by 29
  34. 2024TexGaussian: Generating High-quality PBR Material via Octree-based 3D Gaussian Splattingcited by 28
  35. 2025MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusioncited by 28
  36. 2024MaPa: Text-driven Photorealistic Material Painting for 3D Shapescited by 27
  37. 2025BlenderGym: Benchmarking Foundational Model Systems for Graphics Editingcited by 27
  38. 2024SyncTweedies: A General Generative Framework Based on Synchronized Diffusionscited by 26

50 most cited of 234 All 234 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation
  2. 2024Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis
  3. 2026UltraTex: Unleashing 2K Multi-View Diffusion for 3D Texturing
  4. 2023Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields
  5. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Models
  6. 2023Text-guided High-definition Consistency Texture Model
  7. 2025InsTex: Indoor Scenes Stylized Texture Synthesis
  8. 20243DTopia: Large Text-to-3D Generation Model with Hybrid Diffusion Priors