# TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion

arXiv: https://arxiv.org/abs/2401.09416 · HTML: https://mlsanity.com/paper/2401.09416/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 70 references that are in the map.
- First posted on arXiv 17 Jan 2024.

## Builds on (most cited first, 30 of 59)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model](https://mlsanity.com/paper/2310.15110/) (2023)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images](https://mlsanity.com/paper/2209.11163/) (2022)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)

## Built on by (most cited first, 30 of 52)

- [Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation](https://mlsanity.com/paper/2501.12202/) (2025)
- [Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Details](https://mlsanity.com/paper/2506.16504/) (2025)
- [TEXGen: a Generative Diffusion Model for Mesh Textures](https://mlsanity.com/paper/2411.14740/) (2024)
- [MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D](https://mlsanity.com/paper/2411.02336/) (2024)
- [A Survey on Personalized Content Synthesis with Diffusion Models](https://mlsanity.com/paper/2405.05538/) (2024)
- [ColorPeel: Color Prompt Learning with Diffusion Models via Color and Shape Disentanglement](https://mlsanity.com/paper/2407.07197/) (2024)
- [TexGaussian: Generating High-quality PBR Material via Octree-based 3D Gaussian Splatting](https://mlsanity.com/paper/2411.19654/) (2024)
- [MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusion](https://mlsanity.com/paper/2503.10289/) (2025)
- [Garment3DGen: 3D Garment Stylization and Texture Generation](https://mlsanity.com/paper/2403.18816/) (2024)
- [ZeST: Zero-Shot Material Transfer from a Single Image](https://mlsanity.com/paper/2404.06425/) (2024)
- [Collaborative Control for Geometry-Conditioned PBR Image Generation](https://mlsanity.com/paper/2402.05919/) (2024)
- [Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketches](https://mlsanity.com/paper/2408.04567/) (2024)
- [MatAtlas: Text-driven Consistent Geometry Texturing and Material Assignment](https://mlsanity.com/paper/2404.02899/) (2024)
- [RoomTex: Texturing Compositional Indoor Scenes via Iterative Inpainting](https://mlsanity.com/paper/2406.02461/) (2024)
- [StyleTex: Style Image-Guided Texture Generation for 3D Models](https://mlsanity.com/paper/2411.00399/) (2024)
- [Make-A-Texture: Fast Shape-Aware Texture Generation in 3 Seconds](https://mlsanity.com/paper/2412.07766/) (2024)
- [Diffusion Models in 3D Vision: A Survey](https://mlsanity.com/paper/2410.04738/) (2024)
- [DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generation](https://mlsanity.com/paper/2511.23127/) (2025)
- [MaterialFusion: High-Quality, Zero-Shot, and Controllable Material Transfer with Diffusion Models](https://mlsanity.com/paper/2502.06606/) (2025)
- [VideoMat: Extracting PBR Materials from Video Diffusion Models](https://mlsanity.com/paper/2506.09665/) (2025)
- [MARBLE: Material Recomposition and Blending in CLIP-Space](https://mlsanity.com/paper/2506.05313/) (2025)
- [FlexiTex: Enhancing Texture Generation via Visual Guidance](https://mlsanity.com/paper/2409.12431/) (2024)
- [FlexPainter: Flexible and Multi-View Consistent Texture Generation](https://mlsanity.com/paper/2506.02620/) (2025)
- [UrbanCAD: Towards Highly Controllable and Photorealistic 3D Vehicles for Urban Scene Simulation](https://mlsanity.com/paper/2411.19292/) (2024)
- [SeqTex: Generate Mesh Textures in Video Sequence](https://mlsanity.com/paper/2507.04285/) (2025)
- [GAP: Gaussianize Any Point Clouds with Text Guidance](https://mlsanity.com/paper/2508.05631/) (2025)
- [GECO: Generative Image-to-3D within a SECOnd](https://mlsanity.com/paper/2405.20327/) (2024)
- [LineArt: A Knowledge-guided Training-free High-quality Appearance Transfer for Design Drawing with Diffusion Model](https://mlsanity.com/paper/2412.11519/) (2024)
- [Jointly Generating Multi-view Consistent PBR Textures using Collaborative Control](https://mlsanity.com/paper/2410.06985/) (2024)
- [TactStyle: Generating Tactile Textures with Generative AI for Digital Fabrication](https://mlsanity.com/paper/2503.02007/) (2025)

## Abstract (arXiv)

We present TextureDreamer, a novel image-guided texture synthesis method to transfer relightable textures from a small number of input images (3 to 5) to target 3D shapes across arbitrary categories. Texture creation is a pivotal challenge in vision and graphics. Industrial companies hire experienced artists to manually craft textures for 3D assets. Classical methods require densely sampled views and accurately aligned geometry, while learning-based methods are confined to category-specific shapes within the dataset. In contrast, TextureDreamer can transfer highly detailed, intricate textures from real-world environments to arbitrary objects with only a few casually captured images, potentially significantly democratizing texture creation. Our core idea, personalized geometry-aware score distillation (PGSD), draws inspiration from recent advancements in diffuse models, including personalized modeling for texture information extraction, variational score distillation for detailed appearance synthesis, and explicit geometry guidance with ControlNet. Our integration and several essential modifications substantially improve the texture quality. Experiments on real images spanning different categories show that TextureDreamer can successfully transfer highly realistic, semantic meaningful texture to arbitrary objects, surpassing the visual quality of previous state-of-the-art.