# TexGaussian: Generating High-quality PBR Material via Octree-based 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2411.19654 · HTML: https://mlsanity.com/paper/2411.19654/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 61 references that are in the map.
- First posted on arXiv 29 Nov 2024.

## Builds on (most cited first, 30 of 52)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation](https://mlsanity.com/paper/2402.05054/) (2024)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [CLIP-Mesh: Generating textured meshes from text using pretrained image-text models](https://mlsanity.com/paper/2203.13333/) (2022)
- [GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces](https://mlsanity.com/paper/2311.17977/) (2023)

## Built on by (most cited first, 28 of 28)

- [MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusion](https://mlsanity.com/paper/2503.10289/) (2025)
- [UniTEX: Universal High Fidelity Generative Texturing for 3D Shapes](https://mlsanity.com/paper/2505.23253/) (2025)
- [RomanTex: Decoupling 3D-aware Rotary Positional Embedded Multi-Attention Network for Texture Synthesis](https://mlsanity.com/paper/2503.19011/) (2025)
- [CoRe3D: Collaborative Reasoning as a Foundation for 3D Intelligence](https://mlsanity.com/paper/2512.12768/) (2025)
- [Auto-Regressively Generating Multi-View Consistent Images](https://mlsanity.com/paper/2506.18527/) (2025)
- [MuMA: 3D PBR Texturing via Multi-Channel Multi-View Generation and Agentic Post-Processing](https://mlsanity.com/paper/2503.18461/) (2025)
- [NaTex: Seamless Texture Generation as Latent Color Diffusion](https://mlsanity.com/paper/2511.16317/) (2025)
- [MVInverse: Feed-forward Multi-view Inverse Rendering in Seconds](https://mlsanity.com/paper/2512.21003/) (2025)
- [Advancing high-fidelity 3D and Texture Generation with 2.5D latents](https://mlsanity.com/paper/2505.21050/) (2025)
- [LaFiTe: A Generative Latent Field for 3D Native Texturing](https://mlsanity.com/paper/2512.04786/) (2025)
- [CaliTex: Geometry-Calibrated Attention for View-Coherent 3D Texture Generation](https://mlsanity.com/paper/2511.21309/) (2025)
- [PacTure: Efficient PBR Texture Generation on Packed Views with Visual Autoregressive Models](https://mlsanity.com/paper/2505.22394/) (2025)
- [Large Material Gaussian Model for Relightable 3D Generation](https://mlsanity.com/paper/2509.22112/) (2025)
- [Material-informed Gaussian Splatting for 3D World Reconstruction in a Digital Twin](https://mlsanity.com/paper/2511.20348/) (2025)
- [Photo3D: Advancing Photorealistic 3D Generation through Structure-Aligned Detail Enhancement](https://mlsanity.com/paper/2512.08535/) (2025)
- [Hitem3D 2.0: Multi-View Guided Native 3D Texture Generation](https://mlsanity.com/paper/2604.09231/) (2026)
- [Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Data](https://mlsanity.com/paper/2604.13688/) (2026)
- [ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation](https://mlsanity.com/paper/2607.06565/) (2026)
- [Does Native 3D Texture Generation Necessarily Require 3D Assets for Training?](https://mlsanity.com/paper/2609.34621/) (2026)
- [MatLat: Material Latent Space for PBR Texture Generation](https://mlsanity.com/paper/2512.17302/) (2025)
- [GaussianGrow: Geometry-aware Gaussian Growing from 3D Point Clouds with Text Guidance](https://mlsanity.com/paper/2604.05721/) (2026)
- [DreamLifting: A Plug-in Module Lifting MV Diffusion Models for 3D Asset Generation](https://mlsanity.com/paper/2509.07435/) (2025)
- [TexSpot: 3D Texture Enhancement with Spatially-uniform Point Latent Representation](https://mlsanity.com/paper/2602.12157/) (2026)
- [GarmentPainter: Efficient 3D Garment Texture Synthesis with Character-Guided Diffusion Model](https://mlsanity.com/paper/2603.08228/) (2026)
- [Advances in Neural 3D Mesh Texturing: A Survey](https://mlsanity.com/paper/2606.00137/) (2026)
- [Text-based Tactile Graphics Generation for the Visually Impaired](https://mlsanity.com/paper/2607.22674/) (2026)
- [Luce: Relightable Gaussians for 3D Asset Generation](https://mlsanity.com/paper/2608.23943/) (2026)
- [TaoTex: Boosting Texture Detail Fidelity for Native 3D Material Generation](https://mlsanity.com/paper/2609.34934/) (2026)

## Abstract (arXiv)

Physically Based Rendering (PBR) materials play a crucial role in modern graphics, enabling photorealistic rendering across diverse environment maps. Developing an effective and efficient algorithm that is capable of automatically generating high-quality PBR materials rather than RGB texture for 3D meshes can significantly streamline the 3D content creation. Most existing methods leverage pre-trained 2D diffusion models for multi-view image synthesis, which often leads to severe inconsistency between the generated textures and input 3D meshes. This paper presents TexGaussian, a novel method that uses octant-aligned 3D Gaussian Splatting for rapid PBR material generation. Specifically, we place each 3D Gaussian on the finest leaf node of the octree built from the input 3D mesh to render the multi-view images not only for the albedo map but also for roughness and metallic. Moreover, our model is trained in a regression manner instead of diffusion denoising, capable of generating the PBR material for a 3D mesh in a single feed-forward process. Extensive experiments on publicly available benchmarks demonstrate that our method synthesizes more visually pleasing PBR materials and runs faster than previous methods in both unconditional and text-conditional scenarios, exhibiting better consistency with the given geometry. Our code and trained models are available at https://3d-aigc.github.io/TexGaussian.