# Learning to Predict Indoor Illumination from a Single Image

arXiv: https://arxiv.org/abs/1704.00090 · HTML: https://mlsanity.com/paper/1704.00090/

## Key facts

- Cited by 161 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2017.

## Built on by (most cited first, 30 of 161)

- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021)
- [Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing](https://mlsanity.com/paper/2211.03017/) (2022)
- [DiffusionLight: Light Probes for Free by Painting a Chrome Ball](https://mlsanity.com/paper/2312.09168/) (2023)
- [Making Images Real Again: A Comprehensive Survey on Deep Image Composition](https://mlsanity.com/paper/2106.14490/) (2021)
- [Physically-Based Editing of Indoor Scene Lighting from a Single Image](https://mlsanity.com/paper/2205.09343/) (2022)
- [IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes](https://mlsanity.com/paper/2206.08423/) (2022)
- [StyleLight: HDR Panorama Generation for Lighting Estimation and Editing](https://mlsanity.com/paper/2207.14811/) (2022)
- [DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderer](https://mlsanity.com/paper/2111.00140/) (2021)
- [PanoDiffusion: 360-degree Panorama Outpainting via Diffusion](https://mlsanity.com/paper/2307.03177/) (2023)
- [Free-viewpoint Indoor Neural Relighting from Multi-view Stereo](https://mlsanity.com/paper/2106.13299/) (2021)
- [Diverse Plausible 360-Degree Image Outpainting for Efficient 3DCG Background Creation](https://mlsanity.com/paper/2203.14668/) (2022)
- [Neural Video Portrait Relighting in Real-time via Consistency Modeling](https://mlsanity.com/paper/2104.00484/) (2021)
- [360-Degree Panorama Generation from Few Unregistered NFoV Images](https://mlsanity.com/paper/2308.14686/) (2023)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion](https://mlsanity.com/paper/2410.24203/) (2024)
- [Shadow Generation for Composite Image in Real-world Scenes](https://mlsanity.com/paper/2104.10338/) (2021)
- [Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion](https://mlsanity.com/paper/2208.09480/) (2022)
- [A Dataset of Relighted 3D Interacting Hands](https://mlsanity.com/paper/2310.17768/) (2023)
- [CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation](https://mlsanity.com/paper/2501.17162/) (2025)
- [Geometry-aware Single-image Full-body Human Relighting](https://mlsanity.com/paper/2207.04750/) (2022)
- [Cafca: High-quality Novel View Synthesis of Expressive Faces from Casual Few-shot Captures](https://mlsanity.com/paper/2410.00630/) (2024)
- [Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generation](https://mlsanity.com/paper/2309.03467/) (2023)
- [Guided Co-Modulated GAN for 360deg Field of View Extrapolation](https://mlsanity.com/paper/2204.07286/) (2022)
- [EverLight: Indoor-Outdoor Editable HDR Lighting Estimation](https://mlsanity.com/paper/2304.13207/) (2023)
- [Single-image Full-body Human Relighting](https://mlsanity.com/paper/2107.07259/) (2021)
- [GMLight: Lighting Estimation via Geometric Distribution Approximation](https://mlsanity.com/paper/2102.10244/) (2021)
- [MaterialFusion: Enhancing Inverse Rendering with Material Diffusion Priors](https://mlsanity.com/paper/2409.15273/) (2024)
- [Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering](https://mlsanity.com/paper/2408.09702/) (2024)
- [Lite2Relight: 3D-aware Single Image Portrait Relighting](https://mlsanity.com/paper/2407.10487/) (2024)

## Abstract (arXiv)

We propose an automatic method to infer high dynamic range illumination from a single, limited field-of-view, low dynamic range photograph of an indoor scene. In contrast to previous work that relies on specialized image capture, user input, and/or simple scene models, we train an end-to-end deep neural network that directly regresses a limited field-of-view photo to HDR illumination, without strong assumptions on scene geometry, material properties, or lighting. We show that this can be accomplished in a three step process: 1) we train a robust lighting classifier to automatically annotate the location of light sources in a large dataset of LDR environment maps, 2) we use these annotations to train a deep neural network that predicts the location of lights in a scene from a single limited field-of-view photo, and 3) we fine-tune this network using a small dataset of HDR environment maps to predict light intensities. This allows us to automatically recover high-quality HDR illumination estimates that significantly outperform previous state-of-the-art methods. Consequently, using our illumination estimates for applications like 3D object insertion, we can achieve results that are photo-realistic, which is validated via a perceptual user study.