mlsanityData to 7 Oct 2026

Texture Space Material Diffusion

arXiv 2609.37654 ↗First posted 29 Sep 2026cs.CV cs.GRcited by 0

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present a method for generating high quality materials for 3D objects entirely in texture space. We finetune a video diffusion transformer for text-guided material generation, multi-view material generation, and material upscaling. Our key insight is to use the known projection from image space to texture space, enabling the diffusion process to generalize across arbitrary geometries and texture parameterizations. This approach also avoids the view consistency issues inherent in video and multi-view diffusion models. Because texture space is two dimensional, we can reuse the strong priors of pretrained video diffusion models. We apply our method to high quality material reconstruction from posed photos captured under unknown lighting, as well as to text- and image guided material generation. Our method can scale to high resolutions (8K), 100+ input views, and neural material representations. In quantitative and qualitative evaluations we show state-of-the-art results for material generation and reconstruction.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 77 of its 89 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

77 of 89 references.

Closest ancestors · 26

Same direction, newest first

  1. 2026RGBX-Next: Towards Realistic Generative Rendering from G-Buffers14 Aug
  2. 2026VideoMatGen: PBR Materials through Joint Generative Modeling17 Mar
  3. 2025SViM3D: Stable Video Material Diffusion for Single Image 3D Generation9 Oct
  4. 2025Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images12 Sep
  5. 2025RomanTex: Decoupling 3D-aware Rotary Positional Embedded Multi-Attention Network for Texture Synthesis24 Mar
  6. 2025Generative Detail Enhancement for Physically Based Materials19 Feb
  7. 2025DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models30 Jan
  8. 2024Material Anything: Generating Materials for Any 3D Object via Diffusion22 Nov
  9. 2024TEXGen: a Generative Diffusion Model for Mesh Textures22 Nov
  10. 2024MaterialFusion: Enhancing Inverse Rendering with Material Diffusion Priors23 Sep
  11. 2024Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Models11 Sep
  12. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets30 May
14 more
  1. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models27 May
  2. 2024RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion models1 May
  3. 2024MaPa: Text-driven Photorealistic Material Painting for 3D Shapes26 Apr
  4. 2024IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination17 Apr
  5. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  6. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models21 Dec
  7. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
  8. 2023ControlMat: A Controlled Generative Approach to Material Capture4 Sep
  9. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  10. 2023HiFA: High-fidelity Text-to-3D Generation with Advanced Diffusion Guidance30 May
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation25 May
  12. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Models20 Mar
  13. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  14. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 51

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  6. 2022Scalable Diffusion Models with Transformerscited by 5,931
  7. 2024Qwen2.5 Technical Reportcited by 4,713
  8. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  12. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
38 more
  1. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  2. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  3. 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 742
  4. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  5. 2025VACE: All-in-One Video Creation and Editingcited by 467
  6. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
  7. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  8. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 344
  9. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
  10. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  11. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  12. 2025Hunyuan3D 2.1: From Images to High-Fidelity 3D Assets with Production-Ready PBR Materialcited by 157
  13. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  14. 2020Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipelinecited by 131
  15. 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 103
  16. 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 100
  17. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  18. 2023Intrinsic Image Diffusion for Indoor Single-view Material Estimationcited by 73
  19. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  20. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  21. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
  22. 2025TexVerse: A Universe of 3D Objects with High-Resolution Texturescited by 41
  23. 2025Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Datasetcited by 29
  24. 2025MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusioncited by 28
  25. 2017Normal Integration: A Surveycited by 21
  26. 2025Positional Encoding Fieldcited by 20
  27. 2024IRIS: Inverse Rendering of Indoor Scenes from Low Dynamic Range Imagescited by 19
  28. 2025IntrinsiX: High-Quality PBR Generation using Image Priorscited by 18
  29. 2024Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materialscited by 17
  30. 2024MatAtlas: Text-driven Consistent Geometry Texturing and Material Assignmentcited by 14
  31. 2025Pandora3D: A Comprehensive Framework for High-Quality 3D Shape and Texture Generationcited by 14
  32. 2026Pixal3D: Pixel-Aligned 3D Generation from Imagescited by 11
  33. 2025MVPainter: Accurate and Detailed 3D Texture Generation via Multi-View Diffusion with Geometric Controlcited by 7
  34. 2025Intrinsic Image Fusion for Multi-View 3D Material Reconstructioncited by 7
  35. 2025PBR-SR: Mesh PBR Texture Super Resolution from 2D Image Priorscited by 4
  36. 2025MatPedia: A Universal Generative Foundation for High-Fidelity Material Synthesiscited by 3
  37. 2026VideoNeuMat: Neural Material Extraction from Generative Video Modelscited by 2
  38. 2026LSRM: High-Fidelity Object-Centric Reconstruction via Scaled Context Windowscited by 2

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MatMart: Material Reconstruction of 3D Objects via Diffusion
  2. 2024MaterialPicker: Multi-Modal DiT-Based Material Generation
  3. 2025RealMat: Realistic Materials with Diffusion and Reinforcement Learning
  4. 2024GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation
  5. 2025StableIntrinsic: Detail-preserving One-step Diffusion Model for Multi-view Material Estimation
  6. 2024MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets
  7. 2025VideoMat: Extracting PBR Materials from Video Diffusion Models
  8. 2023Text-Guided Texturing by Synchronized Multi-View Diffusion