Seeing the World in a Bag of Chips
Not on the map. Why?
Abstract (arXiv)
We address the dual problems of novel view synthesis and environment reconstruction from hand-held RGBD sensors. Our contributions include 1) modeling highly specular objects, 2) modeling inter-reflections and Fresnel effects, and 3) enabling surface light field reconstruction with the same input needed to reconstruct shape alone. In cases where scene surface has a strong mirror-like material component, we generate highly detailed environment images, revealing room composition, objects, people, buildings, and trees visible through windows. Our approach yields state of the art view synthesis techniques, operates on low dynamic range imagery, and is robust to geometric and calibration errors.
Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Jan 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 4; 2022: 5; 2023: 11; 2024: 2; 2025: 3.
25 papers.
Sort:
By area:
- 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
- 2023Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracingcited by 138
- 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
- 2022NeILF: Neural Incident Light Field for Physically-based Material Estimationcited by 105
- 2023NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimationcited by 73
- 2023Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflectioncited by 68
- 2023DiffusionLight: Light Probes for Free by Painting a Chrome Ballcited by 63
- 2023Neural Microfacet Fields for Inverse Renderingcited by 37
- 2023NeMF: Inverse Volume Rendering with Neural Microflake Fieldcited by 34
- 2022ORCa: Glossy Objects as Radiance Field Camerascited by 20
- 2023IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracingcited by 20
- 2022SPIDR: SDF-based Neural Point Fields for Illumination and Deformationcited by 17
13 more
- 2023NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fieldscited by 16
- 2023Eclipse: Disambiguating Illumination and Materials using Unintended Shadowscited by 14
- 2024ReCap: Better Gaussian Relighting with Cross-Environment Capturescited by 14
- 2023Accidental Light Probescited by 13
- 2024Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Videocited by 12
- 2025DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpaintingcited by 11
- 2023NeAI: A Pre-convoluted Representation for Plug-and-Play Neural Ambient Illuminationcited by 10
- 2025Materialist: Physically Based Editing Using Single-Image Inverse Renderingcited by 10
- 2022Learning to Learn and Sample BRDFscited by 9
- 2025Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priorscited by 5
- 2022Totems: Physical Objects for Verifying Visual Integritycited by 3
- 2021LIGHTS: LIGHT Specularity Dataset for specular detection in Multi-viewcited by 0
- 2021A Multiple-View Geometric Model for Specularity Prediction on General Curved Surfacescited by 0
- 2025Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priors11 Aug 2025
- 2025DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpainting2 Jul 2025
- 2025Materialist: Physically Based Editing Using Single-Image Inverse Rendering7 Jan 2025
- 2024ReCap: Better Gaussian Relighting with Cross-Environment Captures10 Dec 2024
- 2024Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Video21 Jul 2024
- 2023DiffusionLight: Light Probes for Free by Painting a Chrome Ball14 Dec 2023
- 2023IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing8 Dec 2023
- 2023Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing27 Nov 2023
- 2023NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields22 Sep 2023
- 2023Eclipse: Disambiguating Illumination and Materials using Unintended Shadows25 May 2023
- 2023NeAI: A Pre-convoluted Representation for Plug-and-Play Neural Ambient Illumination18 Apr 2023
- 2023NeMF: Inverse Volume Rendering with Neural Microflake Field3 Apr 2023
13 more
- 2023Neural Microfacet Fields for Inverse Rendering31 Mar 2023
- 2023NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimation30 Mar 2023
- 2023Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection20 Mar 2023
- 2023Accidental Light Probes12 Jan 2023
- 2022ORCa: Glossy Objects as Radiance Field Cameras8 Dec 2022
- 2022SPIDR: SDF-based Neural Point Fields for Illumination and Deformation15 Oct 2022
- 2022Learning to Learn and Sample BRDFs7 Oct 2022
- 2022Totems: Physical Objects for Verifying Visual Integrity26 Sep 2022
- 2022NeILF: Neural Incident Light Field for Physically-based Material Estimation14 Mar 2022
- 2021A Multiple-View Geometric Model for Specularity Prediction on General Curved Surfaces20 Aug 2021
- 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination3 Jun 2021
- 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting1 Apr 2021
- 2021LIGHTS: LIGHT Specularity Dataset for specular detection in Multi-view26 Jan 2021
- 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
- 2023Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracingcited by 138
- 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
- 2022NeILF: Neural Incident Light Field for Physically-based Material Estimationcited by 105
- 2023NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimationcited by 73
- 2023Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflectioncited by 68
- 2023Neural Microfacet Fields for Inverse Renderingcited by 37
- 2023NeMF: Inverse Volume Rendering with Neural Microflake Fieldcited by 34
- 2022ORCa: Glossy Objects as Radiance Field Camerascited by 20
- 2023IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracingcited by 20
- 2022SPIDR: SDF-based Neural Point Fields for Illumination and Deformationcited by 17
- 2023NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fieldscited by 16
12 most cited of 22 All 22 (JSON)
- 2025Materialist: Physically Based Editing Using Single-Image Inverse Rendering7 Jan 2025
- 2024ReCap: Better Gaussian Relighting with Cross-Environment Captures10 Dec 2024
- 2024Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Video21 Jul 2024
- 2023IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing8 Dec 2023
- 2023Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing27 Nov 2023
- 2023NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields22 Sep 2023
- 2023Eclipse: Disambiguating Illumination and Materials using Unintended Shadows25 May 2023
- 2023NeAI: A Pre-convoluted Representation for Plug-and-Play Neural Ambient Illumination18 Apr 2023
- 2023NeMF: Inverse Volume Rendering with Neural Microflake Field3 Apr 2023
- 2023Neural Microfacet Fields for Inverse Rendering31 Mar 2023
- 2023NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimation30 Mar 2023
- 2023Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection20 Mar 2023
12 newest of 22 All 22 (JSON)