# PhotoShape: Photorealistic Materials for Large-Scale Shape Collections

arXiv: https://arxiv.org/abs/1809.09761 · HTML: https://mlsanity.com/paper/1809.09761/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2018.

## Built on by (most cited first, 21 of 21)

- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields](https://mlsanity.com/paper/2112.05139/) (2021)
- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [MVImgNet: A Large-scale Dataset of Multi-view Images](https://mlsanity.com/paper/2303.06042/) (2023)
- [DiffRF: Rendering-Guided 3D Radiance Field Diffusion](https://mlsanity.com/paper/2212.01206/) (2022)
- [SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field](https://mlsanity.com/paper/2303.13277/) (2023)
- [AutoDecoding Latent 3D Diffusion Models](https://mlsanity.com/paper/2307.05445/) (2023)
- [L3DG: Latent 3D Gaussian Diffusion](https://mlsanity.com/paper/2410.13530/) (2024)
- [MaterialFusion: Enhancing Inverse Rendering with Material Diffusion Priors](https://mlsanity.com/paper/2409.15273/) (2024)
- [GINA-3D: Learning to Generate Implicit Neural Assets in the Wild](https://mlsanity.com/paper/2304.02163/) (2023)
- [PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes](https://mlsanity.com/paper/2207.00757/) (2022)
- [EASI-Tex: Edge-Aware Mesh Texturing from Single Image](https://mlsanity.com/paper/2405.17393/) (2024)
- [NeRFs in Robotics: A Survey](https://mlsanity.com/paper/2405.01333/) (2024)
- [Patch-based 3D Natural Scene Generation from a Single Example](https://mlsanity.com/paper/2304.12670/) (2023)
- [Looking 3D: Anomaly Detection with 2D-3D Alignment](https://mlsanity.com/paper/2406.19393/) (2024)
- [Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects](https://mlsanity.com/paper/2209.04183/) (2022)
- [Photo-to-Shape Material Transfer for Diverse Structures](https://mlsanity.com/paper/2205.04018/) (2022)
- [MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models](https://mlsanity.com/paper/2501.15981/) (2025)
- [MaRI: Material Retrieval Integration across Domains](https://mlsanity.com/paper/2503.08111/) (2025)
- [3DGEN: A GAN-based approach for generating novel 3D models from image data](https://mlsanity.com/paper/2312.08094/) (2023)
- [3D-COCO: extension of MS-COCO dataset for image detection and 3D reconstruction modules](https://mlsanity.com/paper/2404.05641/) (2024)

## Abstract (arXiv)

Existing online 3D shape repositories contain thousands of 3D models but lack photorealistic appearance. We present an approach to automatically assign high-quality, realistic appearance models to large scale 3D shape collections. The key idea is to jointly leverage three types of online data -- shape collections, material collections, and photo collections, using the photos as reference to guide assignment of materials to shapes. By generating a large number of synthetic renderings, we train a convolutional neural network to classify materials in real photos, and employ 3D-2D alignment techniques to transfer materials to different parts of each shape model. Our system produces photorealistic, relightable, 3D shapes (PhotoShapes).