# ORCa: Glossy Objects as Radiance Field Cameras

arXiv: https://arxiv.org/abs/2212.04531 · HTML: https://mlsanity.com/paper/2212.04531/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 50 references that are in the map.
- First posted on arXiv 8 Dec 2022.

## Builds on (most cited first, 25 of 25)

- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [NeRFReN: Neural Radiance Fields with Reflections](https://mlsanity.com/paper/2111.15234/) (2021)
- [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021)
- [Fast Spatially-Varying Indoor Lighting Estimation](https://mlsanity.com/paper/1906.03799/) (2019)
- [Lighthouse: Predicting Lighting Volumes for Spatially-Coherent Illumination](https://mlsanity.com/paper/2003.08367/) (2020)
- [IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes](https://mlsanity.com/paper/2206.08423/) (2022)
- [Neural Illumination: Lighting Prediction for Indoor Environments](https://mlsanity.com/paper/1906.07370/) (2019)
- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [Exploring Differential Geometry in Neural Implicits](https://mlsanity.com/paper/2201.09263/) (2022)
- [Seeing the World in a Bag of Chips](https://mlsanity.com/paper/2001.04642/) (2020)
- [What Is Around The Camera?](https://mlsanity.com/paper/1611.09325/) (2016)
- [MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging](https://mlsanity.com/paper/2104.02607/) (2021)
- [Detection and Mapping of Specular Surfaces Using Multibounce Lidar Returns](https://mlsanity.com/paper/2209.03336/) (2022)

## Built on by (most cited first, 20 of 20)

- [NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images](https://mlsanity.com/paper/2305.17398/) (2023)
- [ENVIDR: Implicit Differentiable Renderer with Neural Environment Lighting](https://mlsanity.com/paper/2303.13022/) (2023)
- [CG3D: Compositional Generation for Text-to-3D via Gaussian Splatting](https://mlsanity.com/paper/2311.17907/) (2023)
- [SpecNeRF: Gaussian Directional Encoding for Specular Reflections](https://mlsanity.com/paper/2312.13102/) (2023)
- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields](https://mlsanity.com/paper/2309.13039/) (2023)
- [Neural Radiance Fields for the Real World: A Survey](https://mlsanity.com/paper/2501.13104/) (2025)
- [SUNDIAL: 3D Satellite Understanding through Direct, Ambient, and Complex Lighting Decomposition](https://mlsanity.com/paper/2312.16215/) (2023)
- [3D Motion Magnification: Visualizing Subtle Motions with Time Varying Radiance Fields](https://mlsanity.com/paper/2308.03757/) (2023)
- [Role of Transients in Two-Bounce Non-Line-of-Sight Imaging](https://mlsanity.com/paper/2304.01308/) (2023)
- [Seeing the World through Your Eyes](https://mlsanity.com/paper/2306.09348/) (2023)
- [Geometric implicit neural representations for signed distance functions](https://mlsanity.com/paper/2511.07206/) (2025)
- [Physics-based Indirect Illumination for Inverse Rendering](https://mlsanity.com/paper/2212.04705/) (2022)
- [3D Imaging of Complex Specular Surfaces by Fusing Polarimetric and Deflectometric Information](https://mlsanity.com/paper/2406.01994/) (2024)
- [Event-based Motion-Robust Accurate Shape Estimation for Mixed Reflectance Scenes](https://mlsanity.com/paper/2311.09652/) (2023)
- [PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularization](https://mlsanity.com/paper/2408.06828/) (2024)
- [Can NeRFs See without Cameras?](https://mlsanity.com/paper/2505.22441/) (2025)
- [RefRef: A Dataset and Benchmark for Reconstructing Refractive and Reflective Objects](https://mlsanity.com/paper/2505.05848/) (2025)
- [Shoot-Bounce-3D: Single-Shot Occlusion-Aware 3D from Lidar by Decomposing Two-Bounce Light](https://mlsanity.com/paper/2512.06080/) (2025)
- [Reflection-aware Generative Novel View Synthesis](https://mlsanity.com/paper/2609.05382/) (2026)

## Abstract (arXiv)

Reflections on glossy objects contain valuable and hidden information about the surrounding environment. By converting these objects into cameras, we can unlock exciting applications, including imaging beyond the camera's field-of-view and from seemingly impossible vantage points, e.g. from reflections on the human eye. However, this task is challenging because reflections depend jointly on object geometry, material properties, the 3D environment, and the observer viewing direction. Our approach converts glossy objects with unknown geometry into radiance-field cameras to image the world from the object's perspective. Our key insight is to convert the object surface into a virtual sensor that captures cast reflections as a 2D projection of the 5D environment radiance field visible to the object. We show that recovering the environment radiance fields enables depth and radiance estimation from the object to its surroundings in addition to beyond field-of-view novel-view synthesis, i.e. rendering of novel views that are only directly-visible to the glossy object present in the scene, but not the observer. Moreover, using the radiance field we can image around occluders caused by close-by objects in the scene. Our method is trained end-to-end on multi-view images of the object and jointly estimates object geometry, diffuse radiance, and the 5D environment radiance field.