# Lighting, Reflectance and Geometry Estimation from 360circ Panoramic Stereo

arXiv: https://arxiv.org/abs/2104.09886 · HTML: https://mlsanity.com/paper/2104.09886/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Learning to Predict Indoor Illumination from a Single Image](https://mlsanity.com/paper/1704.00090/) (2017)
- [CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering](https://mlsanity.com/paper/1808.08601/) (2018)
- [Deep Parametric Indoor Lighting Estimation](https://mlsanity.com/paper/1910.08812/) (2019)
- [Fast Spatially-Varying Indoor Lighting Estimation](https://mlsanity.com/paper/1906.03799/) (2019)
- [DeepLight: Learning Illumination for Unconstrained Mobile Mixed Reality](https://mlsanity.com/paper/1904.01175/) (2019)
- [Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination](https://mlsanity.com/paper/2003.08367/) (2020)
- [Neural Illumination: Lighting Prediction for Indoor Environments](https://mlsanity.com/paper/1906.07370/) (2019)
- [Learning Non-Lambertian Object Intrinsics across ShapeNet Categories](https://mlsanity.com/paper/1612.08510/) (2016)
- [360SD-Net: 360deg Stereo Depth Estimation with Learnable Cost Volume](https://mlsanity.com/paper/1911.04460/) (2019)
- [Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression](https://mlsanity.com/paper/1512.02311/) (2015)
- [PointAR: Efficient Lighting Estimation for Mobile Augmented Reality](https://mlsanity.com/paper/2004.00006/) (2020)

## Built on by (most cited first, 11 of 11)

- [360MonoDepth: High-Resolution 360deg Monocular Depth Estimation](https://mlsanity.com/paper/2111.15669/) (2021)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting](https://mlsanity.com/paper/2412.00177/) (2024)
- [Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenes](https://mlsanity.com/paper/2211.10206/) (2022)
- [ScribbleLight: Single Image Indoor Relighting with Scribbles](https://mlsanity.com/paper/2411.17696/) (2024)
- [Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation](https://mlsanity.com/paper/2306.15662/) (2023)
- [Pano-NeRF: Synthesizing High Dynamic Range Novel Views with Geometry from Sparse Low Dynamic Range Panoramic Images](https://mlsanity.com/paper/2312.15942/) (2023)
- [Multi-Camera Lighting Estimation for Photorealistic Front-Facing Mobile Augmented Reality](https://mlsanity.com/paper/2301.06143/) (2023)
- [LitAR: Visually Coherent Lighting for Mobile Augmented Reality](https://mlsanity.com/paper/2301.06184/) (2023)
- [SyncLight: Single-Edit Multi-View Relighting](https://mlsanity.com/paper/2601.16981/) (2026)
- [Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics](https://mlsanity.com/paper/2602.01391/) (2026)

## Abstract (arXiv)

We propose a method for estimating high-definition spatially-varying lighting, reflectance, and geometry of a scene from 360circ stereo images. Our model takes advantage of the 360circ input to observe the entire scene with geometric detail, then jointly estimates the scene's properties with physical constraints. We first reconstruct a near-field environment light for predicting the lighting at any 3D location within the scene. Then we present a deep learning model that leverages the stereo information to infer the reflectance and surface normal. Lastly, we incorporate the physical constraints between lighting and geometry to refine the reflectance of the scene. Both quantitative and qualitative experiments show that our method, benefiting from the 360circ observation of the scene, outperforms prior state-of-the-art methods and enables more augmented reality applications such as mirror-objects insertion.