# NeReF: Neural Refractive Field for Fluid Surface Reconstruction and Implicit Representation

arXiv: https://arxiv.org/abs/2203.04130 · HTML: https://mlsanity.com/paper/2203.04130/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 41 references that are in the map.
- First posted on arXiv 8 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [DeepVoxels: Learning Persistent 3D Feature Embeddings](https://mlsanity.com/paper/1812.01024/) (2018)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Unsupervised Training for 3D Morphable Model Regression](https://mlsanity.com/paper/1806.06098/) (2018)
- [Interpretable Transformations with Encoder-Decoder Networks](https://mlsanity.com/paper/1710.07307/) (2017)
- [Dense Reconstruction of Transparent Objects by Altering Incident Light Paths Through Refraction](https://mlsanity.com/paper/2105.09993/) (2021)

## Built on by (most cited first, 4 of 4)

- [WaterNeRF: Neural Radiance Fields for Underwater Scenes](https://mlsanity.com/paper/2209.13091/) (2022)
- [DerainNeRF: 3D Scene Estimation with Adhesive Waterdrop Removal](https://mlsanity.com/paper/2403.20013/) (2024)
- [Sampling Neural Radiance Fields for Refractive Objects](https://mlsanity.com/paper/2211.14799/) (2022)
- [Inferring Fluid Dynamics via Inverse Rendering](https://mlsanity.com/paper/2304.04446/) (2023)

## Abstract (arXiv)

Existing neural reconstruction schemes such as Neural Radiance Field (NeRF) are largely focused on modeling opaque objects. We present a novel neural refractive field(NeReF) to recover wavefront of transparent fluids by simultaneously estimating the surface position and normal of the fluid front. Unlike prior arts that treat the reconstruction target as a single layer of the surface, NeReF is specifically formulated to recover a volumetric normal field with its corresponding density field. A query ray will be refracted by NeReF according to its accumulated refractive point and normal, and we employ the correspondences and uniqueness of refracted ray for NeReF optimization. We show NeReF, as a global optimization scheme, can more robustly tackle refraction distortions detrimental to traditional methods for correspondence matching. Furthermore, the continuous NeReF representation of wavefront enables view synthesis as well as normal integration. We validate our approach on both synthetic and real data and show it is particularly suitable for sparse multi-view acquisition. We hence build a small light field array and experiment on various surface shapes to demonstrate high fidelity NeReF reconstruction.