# IntrinsiX: High-Quality PBR Generation using Image Priors

arXiv: https://arxiv.org/abs/2504.01008 · HTML: https://mlsanity.com/paper/2504.01008/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 79 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Builds on (most cited first, 30 of 60)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2212.11565/) (2022)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey](https://mlsanity.com/paper/2403.14608/) (2024)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [CAT3D: Create Anything in 3D with Multi-View Diffusion Models](https://mlsanity.com/paper/2405.10314/) (2024)
- [Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks](https://mlsanity.com/paper/2311.06242/) (2023)
- [SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers](https://mlsanity.com/paper/2410.10629/) (2024)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [Text-to-image Diffusion Models in Generative AI: A Survey](https://mlsanity.com/paper/2303.07909/) (2023)

## Built on by (most cited first, 19 of 19)

- [UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting](https://mlsanity.com/paper/2506.15673/) (2025)
- [WorldExplorer: Towards Generating Fully Navigable 3D Scenes](https://mlsanity.com/paper/2506.01799/) (2025)
- [Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assets](https://mlsanity.com/paper/2510.19944/) (2025)
- [VideoMat: Extracting PBR Materials from Video Diffusion Models](https://mlsanity.com/paper/2506.09665/) (2025)
- [Generative World Renderer](https://mlsanity.com/paper/2604.02329/) (2026)
- [OmniX: From Unified Panoramic Generation and Perception to Graphics-Ready 3D Scenes](https://mlsanity.com/paper/2510.26800/) (2025)
- [LaFiTe: A Generative Latent Field for 3D Native Texturing](https://mlsanity.com/paper/2512.04786/) (2025)
- [MatPedia: A Universal Generative Foundation for High-Fidelity Material Synthesis](https://mlsanity.com/paper/2511.16957/) (2025)
- [UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors](https://mlsanity.com/paper/2605.00658/) (2026)
- [Generative Perception of Shape and Material from Differential Motion](https://mlsanity.com/paper/2506.02473/) (2025)
- [MVID: Feed-Forward Multi-View Intrinsic Image Decomposition](https://mlsanity.com/paper/2512.23667/) (2025)
- [Relit-LiVE: Relight Video by Jointly Learning Environment Video](https://mlsanity.com/paper/2605.06658/) (2026)
- [LumiX: Structured and Coherent Text-to-Intrinsic Generation](https://mlsanity.com/paper/2512.02781/) (2025)
- [DiffusionBrowser: Interactive Diffusion Previews via Multi-Branch Decoders](https://mlsanity.com/paper/2512.13690/) (2025)
- [PIXLRelight: Controllable Relighting via Intrinsic Conditioning](https://mlsanity.com/paper/2605.18735/) (2026)
- [GeoStereo: A Unified Stereo Geometry Estimation Framework for Disparity and Surface Normal](https://mlsanity.com/paper/2607.24024/) (2026)
- [Physically Based Rendering in the Latent Space](https://mlsanity.com/paper/2609.21054/) (2026)
- [Texture Space Material Diffusion](https://mlsanity.com/paper/2609.37654/) (2026)
- [MatLoom: Layered Text-to-Material Generation in a Compact Program Space](https://mlsanity.com/paper/2609.40322/) (2026)

## Abstract (arXiv)

We introduce IntrinsiX, a novel method that generates high-quality intrinsic images from text description. In contrast to existing text-to-image models whose outputs contain baked-in scene lighting, our approach predicts physically-based rendering (PBR) maps. This enables the generated outputs to be used for content creation scenarios in core graphics applications that facilitate re-lighting, editing, and texture generation tasks. In order to train our generator, we exploit strong image priors, and pre-train separate models for each PBR material component (albedo, roughness, metallic, normals). We then align these models with a new cross-intrinsic attention formulation that concatenates key and value features in a consistent fashion. This allows us to exchange information between each output modality and to obtain semantically coherent PBR predictions. To ground each intrinsic component, we propose a rendering loss which provides image-space signals to constrain the model, thus facilitating sharp details also in the output BRDF properties. Our results demonstrate detailed intrinsic generation with strong generalization capabilities that outperforms existing intrinsic image decomposition methods used with generated images by a significant margin. Finally, we show a series of applications, including re-lighting, editing, and text-conditioned room-scale PBR texture generation.