mlsanityData to 7 Oct 2026

Jointly Generating Multi-view Consistent PBR Textures using Collaborative Control

arXiv 2410.06985 ↗First posted 9 Oct 2024cs.CV cs.GRcited by 4

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Multi-view consistency remains a challenge for image diffusion models. Even within the Text-to-Texture problem, where perfect geometric correspondences are known a priori, many methods fail to yield aligned predictions across views, necessitating non-trivial fusion methods to incorporate the results onto the original mesh. We explore this issue for a Collaborative Control workflow specifically in PBR Text-to-Texture. Collaborative Control directly models PBR image probability distributions, including normal bump maps; to our knowledge, the only diffusion model to directly output full PBR stacks. We discuss the design decisions involved in making this model multi-view consistent, and demonstrate the effectiveness of our approach in ablation studies, as well as practical applications.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 69 of its 78 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

69 of 78 references.

Closest ancestors · 25

Same direction, newest first

  1. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image30 May
  2. 2024TexPainter: Generative Mesh Texturing with Multi-view Consistency17 May
  3. 2024Multi-view Image Prompted Multi-view Diffusion for Improved 3D Generation26 Apr
  4. 2024MatFusion: A Generative Diffusion Model for SVBRDF Capture24 Apr
  5. 2024IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination17 Apr
  6. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  7. 2023HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D26 Dec
  8. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models21 Dec
  9. 2023UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation14 Dec
  10. 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusion11 Dec
  11. 2023MVDD: Multi-View Depth Diffusion Models8 Dec
  12. 2023Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusion27 Nov
13 more
  1. 2023EucliDreamer: Fast and High-Quality Texturing for 3D Models with Stable Diffusion Depth27 Nov
  2. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model23 Oct
  3. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
  4. 2023TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models20 Oct
  5. 2023Consistent-1-to-3: Consistent Image to 3D View Synthesis via Geometry-aware Diffusion Models4 Oct
  6. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
  7. 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
  8. 2023MatFuse: Controllable Material Generation with Diffusion Models22 Aug
  9. 2023MATLABER: Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR18 Aug
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  11. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Models20 Mar
  12. 2023TEXTure: Text-Guided Texturing of 3D Shapes3 Feb
  13. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2022Generative Adversarial Networkscited by 5,603
  12. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
32 more
  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  3. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  4. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  5. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  6. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  7. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  8. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  9. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  10. 2023MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusioncited by 210
  11. 2024Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attentioncited by 99
  12. 2023Text-Guided Texturing by Synchronized Multi-View Diffusioncited by 86
  13. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  14. 2024IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generationcited by 84
  15. 2023Free3D: Consistent Novel View Synthesis without 3D Representationcited by 72
  16. 2023ConsistNet: Enforcing 3D Consistency for Multi-view Images Diffusioncited by 65
  17. 2024ViewDiff: 3D-Consistent Image Generation with Text-to-Image Modelscited by 57
  18. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  19. 2024Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objectscited by 52
  20. 2024TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusioncited by 51
  21. 2023Instant-3D: Instant Neural Radiance Field Training Towards On-Device AR/VR 3D Reconstructioncited by 49
  22. 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generationcited by 35
  23. 2023Cascade-Zero123: One Image to Highly Consistent 3D with Self-Prompted Nearby Viewscited by 32
  24. 2024Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusioncited by 31
  25. 2023Repaint123: Fast and High-quality One Image to 3D Generation with Progressive Controllable 2D Repaintingcited by 25
  26. 2023Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D priorscited by 24
  27. 2024Collaborative Control for Geometry-Conditioned PBR Image Generationcited by 22
  28. 2024InTeX: Interactive Text-to-texture Synthesis via Unified Depth-aware Inpaintingcited by 22
  29. 2023HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Imagecited by 17
  30. 2023NVS-Adapter: Plug-and-Play Novel View Synthesis from a Single Imagecited by 11
  31. 2021Passing Multi-Channel Material Textures to a 3-Channel Losscited by 6
  32. 2023Consistent Mesh Diffusioncited by 3

Built on by

Cited by year

Citing papers per year: 2025: 4.

4 papers.

  1. 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 120
  2. 2025RomanTex: Decoupling 3D-aware Rotary Positional Embedded Multi-Attention Network for Texture Synthesiscited by 20
  3. 2025PacTure: Efficient PBR Texture Generation on Packed Views with Visual Autoregressive Modelscited by 2
  4. 2025An evaluation of SVBRDF Prediction from Generative Image Models for Appearance Modeling of 3D Scenescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MatLat: Material Latent Space for PBR Texture Generation
  2. 2025Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images
  3. 2024MCMat: Multiview-Consistent and Physically Accurate PBR Material Generation
  4. 2024An Optimization Framework to Enforce Multi-View Consistency for Texturing 3D Meshes
  5. 2024RoCoTex: A Robust Method for Consistent Texture Synthesis with Diffusion Models
  6. 2024DreamPBR: Text-driven Generation of High-resolution SVBRDF with Multi-modal Guidance
  7. 2025MuMA: 3D PBR Texturing via Multi-Channel Multi-View Generation and Agentic Post-Processing
  8. 2024GenesisTex2: Stable, Consistent and High-Quality Text-to-Texture Generation