# Eikonal Fields for Refractive Novel-View Synthesis

arXiv: https://arxiv.org/abs/2202.00948 · HTML: https://mlsanity.com/paper/2202.00948/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 67 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Builds on (most cited first, 25 of 25)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)
- [Neural Volumes: Learning Dynamic Renderable Volumes from Images](https://mlsanity.com/paper/1906.07751/) (2019)
- [Escaping Plato's Cave: 3D Shape From Adversarial Rendering](https://mlsanity.com/paper/1811.11606/) (2018)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation](https://mlsanity.com/paper/1910.02550/) (2019)
- [NeRFReN: Neural Radiance Fields with Reflections](https://mlsanity.com/paper/2111.15234/) (2021)
- [Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects](https://mlsanity.com/paper/2110.14217/) (2021)
- [EikoNet: Solving the Eikonal equation with Deep Neural Networks](https://mlsanity.com/paper/2004.00361/) (2020)
- [Learning Neural Light Fields with Ray-Space Embedding Networks](https://mlsanity.com/paper/2112.01523/) (2021)
- [Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes](https://mlsanity.com/paper/2004.10904/) (2020)
- [X-Fields: Implicit Neural View-, Light- and Time-Image Interpolation](https://mlsanity.com/paper/2010.00450/) (2020)
- [Full 3D Reconstruction of Transparent Objects](https://mlsanity.com/paper/1805.03482/) (2018)
- [Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects](https://mlsanity.com/paper/2009.09144/) (2020)
- [TRANSPR: Transparency Ray-Accumulating Neural 3D Scene Point Renderer](https://mlsanity.com/paper/2009.02819/) (2020)
- [MirrorNeRF: One-shot Neural Portrait Radiance Field from Multi-mirror Catadioptric Imaging](https://mlsanity.com/paper/2104.02607/) (2021)
- [Computing Light Transport Gradients using the Adjoint Method](https://mlsanity.com/paper/2006.15059/) (2020)
- [Learning Transparent Object Matting](https://mlsanity.com/paper/1907.11544/) (2019)

## Built on by (most cited first, 30 of 30)

- [4D-Rotor Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes](https://mlsanity.com/paper/2402.03307/) (2024)
- [HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling](https://mlsanity.com/paper/2301.02238/) (2023)
- [Ref-GS: Directional Factorization for 2D Gaussian Splatting](https://mlsanity.com/paper/2412.00905/) (2024)
- [Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic Solids](https://mlsanity.com/paper/2405.02386/) (2024)
- [NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields](https://mlsanity.com/paper/2309.13039/) (2023)
- [TransparentGS: Fast Inverse Rendering of Transparent Objects with Gaussians](https://mlsanity.com/paper/2504.18768/) (2025)
- [NEMTO: Neural Environment Matting for Novel View and Relighting Synthesis of Transparent Objects](https://mlsanity.com/paper/2303.11963/) (2023)
- [OmnimatteRF: Robust Omnimatte with 3D Background Modeling](https://mlsanity.com/paper/2309.07749/) (2023)
- [SpectralNeRF: Physically Based Spectral Rendering with Neural Radiance Field](https://mlsanity.com/paper/2312.08692/) (2023)
- [Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variation](https://mlsanity.com/paper/2412.15211/) (2024)
- [Plateau-reduced Differentiable Path Tracing](https://mlsanity.com/paper/2211.17263/) (2022)
- [NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing](https://mlsanity.com/paper/2303.11219/) (2023)
- [NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics](https://mlsanity.com/paper/2411.10189/) (2024)
- [Single View Refractive Index Tomography with Neural Fields](https://mlsanity.com/paper/2309.04437/) (2023)
- [Flying with Photons: Rendering Novel Views of Propagating Light](https://mlsanity.com/paper/2404.06493/) (2024)
- [Sampling Neural Radiance Fields for Refractive Objects](https://mlsanity.com/paper/2211.14799/) (2022)
- [MS-NeRF: Multi-Space Neural Radiance Fields](https://mlsanity.com/paper/2305.04268/) (2023)
- [SeamlessNeRF: Stitching Part NeRFs with Gradient Propagation](https://mlsanity.com/paper/2311.16127/) (2023)
- [NeuroPump: Simultaneous Geometric and Color Rectification for Underwater Images](https://mlsanity.com/paper/2412.15890/) (2024)
- [Inferring Fluid Dynamics via Inverse Rendering](https://mlsanity.com/paper/2304.04446/) (2023)
- [Surf-NeRF: Surface Regularised Neural Radiance Fields](https://mlsanity.com/paper/2411.18652/) (2024)
- [PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularization](https://mlsanity.com/paper/2408.06828/) (2024)
- [RT-Splatting: Joint Reflection-Transmission Modeling with Gaussian Splatting](https://mlsanity.com/paper/2605.18263/) (2026)
- [Neural Radiance Fields for Transparent Object Using Visual Hull](https://mlsanity.com/paper/2312.08118/) (2023)
- [RefRef: A Dataset and Benchmark for Reconstructing Refractive and Reflective Objects](https://mlsanity.com/paper/2505.05848/) (2025)
- [Through the Curved Cover: Synthesizing Cover Aberrated Scenes with Refractive Field](https://mlsanity.com/paper/2411.06365/) (2024)
- [DiffTrans: Differentiable Geometry-Materials Decomposition for Reconstructing Transparent Objects](https://mlsanity.com/paper/2603.00413/) (2026)
- [RefracGS: Novel View Synthesis Through Refractive Water Surfaces with 3D Gaussian Ray Tracing](https://mlsanity.com/paper/2603.21695/) (2026)
- [GLINT: Modeling Scene-Scale Transparency via Gaussian Radiance Transport](https://mlsanity.com/paper/2603.26181/) (2026)
- [Light Field Primitive for Novel View Synthesis](https://mlsanity.com/paper/2609.31198/) (2026)

## Abstract (arXiv)

We tackle the problem of generating novel-view images from collections of 2D images showing refractive and reflective objects. Current solutions assume opaque or transparent light transport along straight paths following the emission-absorption model. Instead, we optimize for a field of 3D-varying Index of Refraction (IoR) and trace light through it that bends toward the spatial gradients of said IoR according to the laws of eikonal light transport.