# Seeing the World in a Bag of Chips

arXiv: https://arxiv.org/abs/2001.04642 · HTML: https://mlsanity.com/paper/2001.04642/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2020.

## Built on by (most cited first, 25 of 25)

- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing](https://mlsanity.com/paper/2311.16043/) (2023)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022)
- [NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimation](https://mlsanity.com/paper/2303.17147/) (2023)
- [Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflection](https://mlsanity.com/paper/2303.10840/) (2023)
- [DiffusionLight: Light Probes for Free by Painting a Chrome Ball](https://mlsanity.com/paper/2312.09168/) (2023)
- [Neural Microfacet Fields for Inverse Rendering](https://mlsanity.com/paper/2303.17806/) (2023)
- [NeMF: Inverse Volume Rendering with Neural Microflake Field](https://mlsanity.com/paper/2304.00782/) (2023)
- [ORCa: Glossy Objects as Radiance Field Cameras](https://mlsanity.com/paper/2212.04531/) (2022)
- [IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing](https://mlsanity.com/paper/2312.05210/) (2023)
- [SPIDR: SDF-based Neural Point Fields for Illumination and Deformation](https://mlsanity.com/paper/2210.08398/) (2022)
- [NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields](https://mlsanity.com/paper/2309.13039/) (2023)
- [Eclipse: Disambiguating Illumination and Materials using Unintended Shadows](https://mlsanity.com/paper/2305.16321/) (2023)
- [ReCap: Better Gaussian Relighting with Cross-Environment Captures](https://mlsanity.com/paper/2412.07534/) (2024)
- [Accidental Light Probes](https://mlsanity.com/paper/2301.05211/) (2023)
- [Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Video](https://mlsanity.com/paper/2407.15212/) (2024)
- [DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpainting](https://mlsanity.com/paper/2507.01305/) (2025)
- [NeAI: A Pre-convoluted Representation for Plug-and-Play Neural Ambient Illumination](https://mlsanity.com/paper/2304.08757/) (2023)
- [Materialist: Physically Based Editing Using Single-Image Inverse Rendering](https://mlsanity.com/paper/2501.03717/) (2025)
- [Learning to Learn and Sample BRDFs](https://mlsanity.com/paper/2210.03510/) (2022)
- [Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priors](https://mlsanity.com/paper/2508.08384/) (2025)
- [Totems: Physical Objects for Verifying Visual Integrity](https://mlsanity.com/paper/2209.13032/) (2022)
- [LIGHTS: LIGHT Specularity Dataset for specular detection in Multi-view](https://mlsanity.com/paper/2101.10772/) (2021)
- [A Multiple-View Geometric Model for Specularity Prediction on General Curved Surfaces](https://mlsanity.com/paper/2108.09378/) (2021)

## Abstract (arXiv)

We address the dual problems of novel view synthesis and environment reconstruction from hand-held RGBD sensors. Our contributions include 1) modeling highly specular objects, 2) modeling inter-reflections and Fresnel effects, and 3) enabling surface light field reconstruction with the same input needed to reconstruct shape alone. In cases where scene surface has a strong mirror-like material component, we generate highly detailed environment images, revealing room composition, objects, people, buildings, and trees visible through windows. Our approach yields state of the art view synthesis techniques, operates on low dynamic range imagery, and is robust to geometric and calibration errors.