# A Real-world Display Inverse Rendering Dataset

arXiv: https://arxiv.org/abs/2508.14411 · HTML: https://mlsanity.com/paper/2508.14411/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 83 references that are in the map.
- First posted on arXiv 20 Aug 2025.

## Builds on (most cited first, 30 of 46)

- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces](https://mlsanity.com/paper/2311.17977/) (2023)
- [GS-IR: 3D Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2311.16473/) (2023)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Modeling Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2204.06837/) (2022)
- [TensoIR: Tensorial Inverse Rendering](https://mlsanity.com/paper/2304.12461/) (2023)
- [NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images](https://mlsanity.com/paper/2305.17398/) (2023)
- [Relightable Gaussian Codec Avatars](https://mlsanity.com/paper/2312.03704/) (2023)
- [NeRF for Outdoor Scene Relighting](https://mlsanity.com/paper/2112.05140/) (2021)
- [IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images](https://mlsanity.com/paper/2204.02232/) (2022)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021)
- [PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereo](https://mlsanity.com/paper/2207.11406/) (2022)
- [IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illumination](https://mlsanity.com/paper/2404.11593/) (2024)
- [IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes](https://mlsanity.com/paper/2206.08423/) (2022)
- [Two-shot Spatially-varying BRDF and Shape Estimation](https://mlsanity.com/paper/2004.00403/) (2020)
- [DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models](https://mlsanity.com/paper/2501.18590/) (2025)
- [Multi-View Photometric Stereo: A Robust Solution and Benchmark Dataset for Spatially Varying Isotropic Materials](https://mlsanity.com/paper/2001.06659/) (2020)
- [Deep Polarization Imaging for 3D shape and SVBRDF Acquisition](https://mlsanity.com/paper/2105.02875/) (2021)
- [PS-FCN: A Flexible Learning Framework for Photometric Stereo](https://mlsanity.com/paper/1807.08696/) (2018)
- [Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark](https://mlsanity.com/paper/2310.16044/) (2023)
- [CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces](https://mlsanity.com/paper/1808.10093/) (2018)
- [GS^3: Efficient Relighting with Triple Gaussian Splatting](https://mlsanity.com/paper/2410.11419/) (2024)
- [ReLight My NeRF: A Dataset for Novel View Synthesis and Relighting of Real World Objects](https://mlsanity.com/paper/2304.10448/) (2023)
- [Relighting Neural Radiance Fields with Shadow and Highlight Hints](https://mlsanity.com/paper/2308.13404/) (2023)
- [Shape and Material Capture at Home](https://mlsanity.com/paper/2104.06397/) (2021)
- [MatFusion: A Generative Diffusion Model for SVBRDF Capture](https://mlsanity.com/paper/2406.06539/) (2024)
- [Scalable, Detailed and Mask-Free Universal Photometric Stereo](https://mlsanity.com/paper/2303.15724/) (2023)

## Built on by (most cited first, 4 of 4)

- [OLATverse: A Large-scale Real-world Object Dataset with Precise Lighting Control](https://mlsanity.com/paper/2511.02483/) (2025)
- [Olbedo: An Albedo and Shading Aerial Dataset for Large-Scale Outdoor Environments](https://mlsanity.com/paper/2602.22025/) (2026)
- [Snapshot Polarimetric Display Inverse Rendering](https://mlsanity.com/paper/2605.24915/) (2026)
- [Ambient-robust Inverse Rendering using Active RGB-NIR Imaging](https://mlsanity.com/paper/2605.30250/) (2026)

## Abstract (arXiv)

Inverse rendering aims to reconstruct geometry and reflectance from captured images. Display-camera imaging systems offer unique advantages for this task: each pixel can easily function as a programmable point light source, and the polarized light emitted by LCD displays facilitates diffuse-specular separation. Despite these benefits, there is currently no public real-world dataset captured using display-camera systems, unlike other setups such as light stages. This absence hinders the development and evaluation of display-based inverse rendering methods. In this paper, we introduce the first real-world dataset for display-based inverse rendering. To achieve this, we construct and calibrate an imaging system comprising an LCD display and stereo polarization cameras. We then capture a diverse set of objects with diverse geometry and reflectance under one-light-at-a-time (OLAT) display patterns. We also provide high-quality ground-truth geometry. Our dataset enables the synthesis of captured images under arbitrary display patterns and different noise levels. Using this dataset, we evaluate the performance of existing photometric stereo and inverse rendering methods, and provide a simple, yet effective baseline for display inverse rendering, outperforming state-of-the-art inverse rendering methods. Code and dataset are available on our project page at https://michaelcsj.github.io/DIR/