# Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priors

arXiv: https://arxiv.org/abs/2508.08384 · HTML: https://mlsanity.com/paper/2508.08384/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 36 of its 48 references that are in the map.
- First posted on arXiv 11 Aug 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [CAT3D: Create Anything in 3D with Multi-View Diffusion Models](https://mlsanity.com/paper/2405.10314/) (2024)
- [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020)
- [ReconFusion: 3D Reconstruction with Diffusion Priors](https://mlsanity.com/paper/2312.02981/) (2023)
- [SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstruction](https://mlsanity.com/paper/2212.00792/) (2022)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content](https://mlsanity.com/paper/1803.02266/) (2018)
- [InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset](https://mlsanity.com/paper/1809.00716/) (2018)
- [Deep Parametric Indoor Lighting Estimation](https://mlsanity.com/paper/1910.08812/) (2019)
- [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021)
- [Perspective Fields for Single Image Camera Calibration](https://mlsanity.com/paper/2212.03239/) (2022)
- [Fast Spatially-Varying Indoor Lighting Estimation](https://mlsanity.com/paper/1906.03799/) (2019)
- [Deep Sky Modeling for Single Image Outdoor Lighting Estimation](https://mlsanity.com/paper/1905.03897/) (2019)
- [DeepLight: Learning Illumination for Unconstrained Mobile Mixed Reality](https://mlsanity.com/paper/1904.01175/) (2019)
- [Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination](https://mlsanity.com/paper/2003.08367/) (2020)
- [IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes](https://mlsanity.com/paper/2206.08423/) (2022)
- [StyleLight: HDR Panorama Generation for Lighting Estimation and Editing](https://mlsanity.com/paper/2207.14811/) (2022)
- [HDR Environment Map Estimation for Real-Time Augmented Reality](https://mlsanity.com/paper/2011.10687/) (2020)
- [GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion](https://mlsanity.com/paper/2412.10209/) (2024)
- [EverLight: Indoor-Outdoor Editable HDR Lighting Estimation](https://mlsanity.com/paper/2304.13207/) (2023)

## Built on by (most cited first, 6 of 6)

- [Lighting in Motion: Spatiotemporal HDR Lighting Estimation](https://mlsanity.com/paper/2512.13597/) (2025)
- [Do Image Editing Models Understand Lighting?](https://mlsanity.com/paper/2606.26738/) (2026)
- [Video Generation Models Are Inherent Lighting Estimators](https://mlsanity.com/paper/2607.04674/) (2026)
- [AEGIR: Modeling Area Emitters for Indoor Inverse Rendering using Gaussian Splatting](https://mlsanity.com/paper/2606.28635/) (2026)
- [RoomLight: A 2.5D Illumination Prior for Indoor Environments](https://mlsanity.com/paper/2609.28300/) (2026)
- [LAMP: 3D Source-Aware Lighting Estimation at Multiple Points](https://mlsanity.com/paper/2610.19266/) (2026)

## Abstract (arXiv)

Indoor lighting estimation from a single image or video remains a challenge due to its highly ill-posed nature, especially when the lighting condition of the scene varies spatially and temporally. We propose a method that estimates from an input video a continuous light field describing the spatiotemporally varying lighting of the scene. We leverage 2D diffusion priors for optimizing such light field represented as a MLP. To enable zero-shot generalization to in-the-wild scenes, we fine-tune a pre-trained image diffusion model to predict lighting at multiple locations by jointly inpainting multiple chrome balls as light probes. We evaluate our method on indoor lighting estimation from a single image or video and show superior performance over compared baselines. Most importantly, we highlight results on spatiotemporally consistent lighting estimation from in-the-wild videos, which is rarely demonstrated in previous works.