# Accidental Light Probes

arXiv: https://arxiv.org/abs/2301.05211 · HTML: https://mlsanity.com/paper/2301.05211/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 62 references that are in the map.
- First posted on arXiv 12 Jan 2023.

## Builds on (most cited first, 30 of 32)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoising](https://mlsanity.com/paper/2206.03380/) (2022)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [Single Image Portrait Relighting](https://mlsanity.com/paper/1905.00824/) (2019)
- [IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images](https://mlsanity.com/paper/2204.02232/) (2022)
- [Neural Inverse Rendering of an Indoor Scene from a Single Image](https://mlsanity.com/paper/1901.02453/) (2019)
- [Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering](https://mlsanity.com/paper/2010.09125/) (2020)
- [InverseRenderNet: Learning single image inverse rendering](https://mlsanity.com/paper/1811.12328/) (2018)
- [Deep Parametric Indoor Lighting Estimation](https://mlsanity.com/paper/1910.08812/) (2019)
- [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021)
- [Fast Spatially-Varying Indoor Lighting Estimation](https://mlsanity.com/paper/1906.03799/) (2019)
- [DeepLight: Learning Illumination for Unconstrained Mobile Mixed Reality](https://mlsanity.com/paper/1904.01175/) (2019)
- [Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination](https://mlsanity.com/paper/2003.08367/) (2020)
- [StyleLight: HDR Panorama Generation for Lighting Estimation and Editing](https://mlsanity.com/paper/2207.14811/) (2022)
- [Neural Illumination: Lighting Prediction for Indoor Environments](https://mlsanity.com/paper/1906.07370/) (2019)
- [EMLight: Lighting Estimation via Spherical Distribution Approximation](https://mlsanity.com/paper/2012.11116/) (2020)
- [Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications](https://mlsanity.com/paper/1904.11176/) (2019)
- [Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion](https://mlsanity.com/paper/2208.09480/) (2022)
- [Seeing the World in a Bag of Chips](https://mlsanity.com/paper/2001.04642/) (2020)
- [Object-based Illumination Estimation with Rendering-aware Neural Networks](https://mlsanity.com/paper/2008.02514/) (2020)
- [Sparse Needlets for Lighting Estimation with Spherical Transport Loss](https://mlsanity.com/paper/2106.13090/) (2021)
- [Learning to Estimate Indoor Lighting from 3D Objects](https://mlsanity.com/paper/1806.03994/) (2018)

## Built on by (most cited first, 13 of 13)

- [DiffusionLight: Light Probes for Free by Painting a Chrome Ball](https://mlsanity.com/paper/2312.09168/) (2023)
- [Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmark](https://mlsanity.com/paper/2310.16044/) (2023)
- [Photorealistic Object Insertion with Diffusion-Guided Inverse Rendering](https://mlsanity.com/paper/2408.09702/) (2024)
- [LightSim: Neural Lighting Simulation for Urban Scenes](https://mlsanity.com/paper/2312.06654/) (2023)
- [LuxDiT: Lighting Estimation with Video Diffusion Transformer](https://mlsanity.com/paper/2509.03680/) (2025)
- [Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variation](https://mlsanity.com/paper/2412.15211/) (2024)
- [DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpainting](https://mlsanity.com/paper/2507.01305/) (2025)
- [Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectance](https://mlsanity.com/paper/2312.04529/) (2023)
- [Materialist: Physically Based Editing Using Single-Image Inverse Rendering](https://mlsanity.com/paper/2501.03717/) (2025)
- [Relighting Scenes with Object Insertions in Neural Radiance Fields](https://mlsanity.com/paper/2406.14806/) (2024)
- [MAIR++: Improving Multi-view Attention Inverse Rendering with Implicit Lighting Representation](https://mlsanity.com/paper/2408.06707/) (2024)
- [MixLight: Borrowing the Best of both Spherical Harmonics and Gaussian Models](https://mlsanity.com/paper/2404.12768/) (2024)
- [Under One Sun: Multi-Object Generative Perception of Materials and Illumination](https://mlsanity.com/paper/2603.19226/) (2026)

## Abstract (arXiv)

Recovering lighting in a scene from a single image is a fundamental problem in computer vision. While a mirror ball light probe can capture omnidirectional lighting, light probes are generally unavailable in everyday images. In this work, we study recovering lighting from accidental light probes (ALPs) -- common, shiny objects like Coke cans, which often accidentally appear in daily scenes. We propose a physically-based approach to model ALPs and estimate lighting from their appearances in single images. The main idea is to model the appearance of ALPs by photogrammetrically principled shading and to invert this process via differentiable rendering to recover incidental illumination. We demonstrate that we can put an ALP into a scene to allow high-fidelity lighting estimation. Our model can also recover lighting for existing images that happen to contain an ALP.