# PBR-Inspired Controllable Diffusion for Image Generation

arXiv: https://arxiv.org/abs/2503.15147 · HTML: https://mlsanity.com/paper/2503.15147/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2025.

## Built on by (most cited first, 3 of 3)

- [IntrinsiX: High-Quality PBR Generation using Image Priors](https://mlsanity.com/paper/2504.01008/) (2025)
- [FrameDiffuser: G-Buffer-Conditioned Diffusion for Neural Forward Frame Rendering](https://mlsanity.com/paper/2512.16670/) (2025)
- [LumiX: Structured and Coherent Text-to-Intrinsic Generation](https://mlsanity.com/paper/2512.02781/) (2025)

## Abstract (arXiv)

Despite recent advances in text-to-image generation, controlling geometric layout and PBR material properties in synthesized scenes remains challenging. We present a pipeline that first produces a G-buffer (albedo, normals, depth, roughness, shading, and metallic) from a text prompt and then renders a final image through a PBR-inspired branch network. This intermediate representation enables fine-grained control: users can copy and paste within specific G-buffer channels to insert or reposition objects, or apply masks to the irradiance channel to adjust lighting locally. As a result, real objects can be seamlessly integrated into virtual scenes. By separating user-friendly scene description from image rendering, our method offers a practical balance between detailed post-generation control and efficient text-driven synthesis. We demonstrate its effectiveness through quantitative evaluations and a user study with 156 participants, showing consistent human preference over strong baselines and confirming that G-buffer control extends the flexibility of text-guided image generation.