# Towards a Perceptual Evaluation Framework for Lighting Estimation

arXiv: https://arxiv.org/abs/2312.04334 · HTML: https://mlsanity.com/paper/2312.04334/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 59 references that are in the map.
- First posted on arXiv 7 Dec 2023.

## Builds on (most cited first, 21 of 21)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [PieAPP: Perceptual Image-Error Assessment through Pairwise Preference](https://mlsanity.com/paper/1806.02067/) (2018)
- [Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild](https://mlsanity.com/paper/2005.13983/) (2020)
- [Deep Parametric Indoor Lighting Estimation](https://mlsanity.com/paper/1910.08812/) (2019)
- [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021)
- [Fast Spatially-Varying Indoor Lighting Estimation](https://mlsanity.com/paper/1906.03799/) (2019)
- [DeepLight: Learning Illumination for Unconstrained Mobile Mixed Reality](https://mlsanity.com/paper/1904.01175/) (2019)
- [StyleLight: HDR Panorama Generation for Lighting Estimation and Editing](https://mlsanity.com/paper/2207.14811/) (2022)
- [Deep Outdoor Illumination Estimation](https://mlsanity.com/paper/1611.06403/) (2016)
- [All-Weather Deep Outdoor Lighting Estimation](https://mlsanity.com/paper/1906.04909/) (2019)
- [HDR-VDP-3: A multi-metric for predicting image differences, quality and contrast distortions in high dynamic range and regular content](https://mlsanity.com/paper/2304.13625/) (2023)
- [Locally Adaptive Structure and Texture Similarity for Image Quality Assessment](https://mlsanity.com/paper/2110.08521/) (2021)
- [EverLight: Indoor-Outdoor Editable HDR Lighting Estimation](https://mlsanity.com/paper/2304.13207/) (2023)
- [Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction](https://mlsanity.com/paper/2304.12372/) (2023)
- [Editable Indoor Lighting Estimation](https://mlsanity.com/paper/2211.03928/) (2022)
- [Spatiotemporally Consistent HDR Indoor Lighting Estimation](https://mlsanity.com/paper/2305.04374/) (2023)

## Built on by (most cited first, 5 of 5)

- [LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting](https://mlsanity.com/paper/2412.00177/) (2024)
- [ZeroComp: Zero-shot Object Compositing from Image Intrinsics via Diffusion](https://mlsanity.com/paper/2410.08168/) (2024)
- [Shedding Light: A Benchmark for Evaluating Lighting Understanding in Generative Image Models](https://mlsanity.com/paper/2609.10787/) (2026)
- [AR as an Evaluation Playground: Bridging Metrics and Visual Perception of Computer Vision Models](https://mlsanity.com/paper/2508.04102/) (2025)
- [Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics](https://mlsanity.com/paper/2602.01391/) (2026)

## Abstract (arXiv)

Progress in lighting estimation is tracked by computing existing image quality assessment (IQA) metrics on images from standard datasets. While this may appear to be a reasonable approach, we demonstrate that doing so does not correlate to human preference when the estimated lighting is used to relight a virtual scene into a real photograph. To study this, we design a controlled psychophysical experiment where human observers must choose their preference amongst rendered scenes lit using a set of lighting estimation algorithms selected from the recent literature, and use it to analyse how these algorithms perform according to human perception. Then, we demonstrate that none of the most popular IQA metrics from the literature, taken individually, correctly represent human perception. Finally, we show that by learning a combination of existing IQA metrics, we can more accurately represent human preference. This provides a new perceptual framework to help evaluate future lighting estimation algorithms.