mlsanityData to 7 Oct 2026

Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models

arXiv 2312.13913 ↗First posted 21 Dec 2023cs.CVcited by 122

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

This paper presents Paint3D, a novel coarse-to-fine generative framework that is capable of producing high-resolution, lighting-less, and diverse 2K UV texture maps for untextured 3D meshes conditioned on text or image inputs. The key challenge addressed is generating high-quality textures without embedded illumination information, which allows the textures to be re-lighted or re-edited within modern graphics pipelines. To achieve this, our method first leverages a pre-trained depth-aware 2D diffusion model to generate view-conditional images and perform multi-view texture fusion, producing an initial coarse texture map. However, as 2D models cannot fully represent 3D shapes and disable lighting effects, the coarse texture map exhibits incomplete areas and illumination artifacts. To resolve this, we train separate UV Inpainting and UVHD diffusion models specialized for the shape-aware refinement of incomplete areas and the removal of illumination artifacts. Through this coarse-to-fine process, Paint3D can produce high-quality 2K UV textures that maintain semantic consistency while being lighting-less, significantly advancing the state-of-the-art in texturing 3D objects.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 59 of its 77 references that are in the map. First posted on arXiv 21 Dec 2023.

Builds on

59 of 77 references.

Closest ancestors · 20

Same direction, newest first

  1. 2023DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior25 Oct
  2. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  3. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models20 Oct
  4. 2023SweetDreamer: Aligning Geometric Priors in 2D Diffusion for Consistent Text-to-3D4 Oct
  5. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  6. 2023HoloFusion: Towards Photo-realistic 3D Generative Modeling28 Aug
  7. 2023IT3D: Improved Text-to-3D Generation with Explicit View Synthesis22 Aug
  8. 2023Texture Generation on 3D Meshes with Point-UV Diffusion21 Aug
  9. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors30 Jun
  10. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  11. 2023TextMesh: Generation of Realistic 3D Meshes From Text Prompts24 Apr
  12. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior24 Mar
8 more
  1. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  2. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Models20 Mar
  3. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  4. 20233DGen: Triplane Latent Diffusion for Textured Mesh Generation9 Mar
  5. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  6. 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
  7. 2022Magic3D: High-Resolution Text-to-3D Content Creation18 Nov
  8. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  11. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  12. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
27 more
  1. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  2. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  3. 2018Demystifying MMD GANscited by 1,268
  4. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  5. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  6. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  7. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  8. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
  9. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  10. 2023Text-to-3D using Gaussian Splattingcited by 299
  11. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  12. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  13. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  14. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  15. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  16. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  17. 2019Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Researchcited by 98
  18. 2022TANGO: Text-driven Photorealistic and Robust 3D Stylization via Lighting Decompositioncited by 83
  19. 2023FocalDreamer: Text-driven 3D Editing via Focal-fusion Assemblycited by 75
  20. 2022Texturify: Generating Textures on 3D Shape Surfacescited by 65
  21. 2022AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesiscited by 39
  22. 2023X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidancecited by 37
  23. 2023Mesh2Tex: Generating Mesh Textures from Image Queriescited by 31
  24. 2020Adversarial Texture Optimization from RGB-D Scanscited by 30
  25. 2023DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagationcited by 25
  26. 2023Enhancing High-Resolution 3D Generation through Pixel-wise Gradient Clippingcited by 17
  27. 2023TUVF: Learning Generalizable Texture UV Radiance Fieldscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 41; 2025: 48; 2026: 33.

122 papers.

Sort:

By area:

  1. 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 342
  2. 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 120
  3. 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 111
  4. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 100
  5. 2024DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generationcited by 93
  6. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 88
  7. 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 85
  8. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  9. 2024Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objectscited by 52
  10. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
  11. 2024An Object is Worth 64x64 Pixels: Generating 3D Object via Image Diffusioncited by 34
  12. 2024MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3Dcited by 34
38 more
  1. 2025EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligencecited by 32
  2. 2024Material Anything: Generating Materials for Any 3D Object via Diffusioncited by 31
  3. 2025MetaScenes: Towards Automated Replica Creation for Real-world 3D Scanscited by 31
  4. 2024TexGaussian: Generating High-quality PBR Material via Octree-based 3D Gaussian Splattingcited by 28
  5. 2025MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusioncited by 28
  6. 2024SyncTweedies: A General Generative Framework Based on Synchronized Diffusionscited by 26
  7. 2024Garment3DGen: 3D Garment Stylization and Texture Generationcited by 26
  8. 2024DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characterscited by 24
  9. 2024UrbanWorld: An Urban World Model for 3D City Generationcited by 23
  10. 2025UniTEX: Universal High Fidelity Generative Texturing for 3D Shapescited by 23
  11. 2024Collaborative Control for Geometry-Conditioned PBR Image Generationcited by 22
  12. 2025RomanTex: Decoupling 3D-aware Rotary Positional Embedded Multi-Attention Network for Texture Synthesiscited by 20
  13. 20243D-Adapter: Geometry-Consistent Multi-View Diffusion for High-Quality 3D Generationcited by 19
  14. 2024BlenderAlchemy: Editing 3D Graphics with Vision-Language Modelscited by 18
  15. 2024RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpaintingcited by 14
  16. 2024StyleTex: Style Image-Guided Texture Generation for 3D Modelscited by 14
  17. 2024Ultraman: Single Image 3D Human Reconstruction with Ultra Speed and Detailcited by 12
  18. 2024Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputscited by 12
  19. 2024Diffusion Models in 3D Vision: A Surveycited by 11
  20. 2024Make-A-Texture: Fast Shape-Aware Texture Generation in 3 Secondscited by 11
  21. 2025VideoMat: Extracting PBR Materials from Video Diffusion Modelscited by 10
  22. 2024TexRO: Generating Delicate Textures of 3D Models by Recursive Optimizationcited by 9
  23. 2024SuperMat: Physically Consistent PBR Material Estimation at Interactive Ratescited by 9
  24. 2024VCD-Texture: Variance Alignment based 3D-2D Co-Denoising for Text-Guided Texturingcited by 8
  25. 2024FlexiTex: Enhancing Texture Generation via Visual Guidancecited by 8
  26. 2024DetailGen3D: Generative 3D Geometry Enhancement via Data-Dependent Flowcited by 8
  27. 2025StochSync: Stochastic Diffusion Synchronization for Image Generation in Arbitrary Spacescited by 8
  28. 2025MuMA: 3D PBR Texturing via Multi-Channel Multi-View Generation and Agentic Post-Processingcited by 8
  29. 2025VideoFrom3D: 3D Scene Video Generation via Complementary Image and Video Diffusion Modelscited by 8
  30. 2025Register Any Point: Scaling 3D Point Cloud Registration by Flow Matchingcited by 8
  31. 2024RGM: Reconstructing High-fidelity 3D Car Assets with Relightable 3D-GS Generative Model from a Single Imagecited by 7
  32. 2024UrbanCAD: Towards Highly Controllable and Photorealistic 3D Vehicles for Urban Scene Simulationcited by 7
  33. 20253D Object Manipulation in a Single Image using Generative Modelscited by 7
  34. 2025FlexPainter: Flexible and Multi-View Consistent Texture Generationcited by 7
  35. 2025SeqTex: Generate Mesh Textures in Video Sequencecited by 7
  36. 2025Elevating 3D Models: High-Quality Texture and Geometry Refinement from a Low-Quality Modelcited by 7
  37. 2025GAP: Gaussianize Any Point Clouds with Text Guidancecited by 7
  38. 2025SViM3D: Stable Video Material Diffusion for Single Image 3D Generationcited by 7

50 most cited of 122 All 122 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Breathing New Life into 3D Assets with Generative Repainting
  2. 2024MaPa: Text-driven Photorealistic Material Painting for 3D Shapes
  3. 2024TEXGen: a Generative Diffusion Model for Mesh Textures
  4. 2023Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repainting
  5. 2024TexPainter: Generative Mesh Texturing with Multi-view Consistency
  6. 2023Consistent Mesh Diffusion
  7. 2025MVPainter: Accurate and Detailed 3D Texture Generation via Multi-View Diffusion with Geometric Control
  8. 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusion