# Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objects

arXiv: https://arxiv.org/abs/2009.09144 · HTML: https://mlsanity.com/paper/2009.09144/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2020.

## Built on by (most cited first, 23 of 23)

- [Multi-View Mesh Reconstruction with Neural Deferred Shading](https://mlsanity.com/paper/2212.04386/) (2022)
- [Eikonal Fields for Refractive Novel-View Synthesis](https://mlsanity.com/paper/2202.00948/) (2022)
- [MEGANE: Morphable Eyeglass and Avatar Network](https://mlsanity.com/paper/2302.04868/) (2023)
- [NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields](https://mlsanity.com/paper/2309.13039/) (2023)
- [TransparentGS: Fast Inverse Rendering of Transparent Objects with Gaussians](https://mlsanity.com/paper/2504.18768/) (2025)
- [Hybrid Mesh-neural Representation for 3D Transparent Object Reconstruction](https://mlsanity.com/paper/2203.12613/) (2022)
- [NEMTO: Neural Environment Matting for Novel View and Relighting Synthesis of Transparent Objects](https://mlsanity.com/paper/2303.11963/) (2023)
- [TSGS: Improving Gaussian Splatting for Transparent Surface Reconstruction via Normal and De-lighting Priors](https://mlsanity.com/paper/2504.12799/) (2025)
- [Seeing Through the Glass: Neural 3D Reconstruction of Object Inside a Transparent Container](https://mlsanity.com/paper/2303.13805/) (2023)
- [NeTO:Neural Reconstruction of Transparent Objects with Self-Occlusion Aware Refraction-Tracing](https://mlsanity.com/paper/2303.11219/) (2023)
- [Polarimetric Inverse Rendering for Transparent Shapes Reconstruction](https://mlsanity.com/paper/2208.11836/) (2022)
- [Mesh Density Adaptation for Template-based Shape Reconstruction](https://mlsanity.com/paper/2307.16205/) (2023)
- [RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation](https://mlsanity.com/paper/2311.12398/) (2023)
- [AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction](https://mlsanity.com/paper/2410.01202/) (2024)
- [Glossy Object Reconstruction with Cost-effective Polarized Acquisition](https://mlsanity.com/paper/2504.07025/) (2025)
- [REF2-NeRF: Reflection and Refraction aware Neural Radiance Field](https://mlsanity.com/paper/2311.17116/) (2023)
- [From Transparent to Opaque: Rethinking Neural Implicit Surfaces with α-NeuS](https://mlsanity.com/paper/2411.05362/) (2024)
- [Neural Radiance Fields for Transparent Object Using Visual Hull](https://mlsanity.com/paper/2312.08118/) (2023)
- [RefRef: A Dataset and Benchmark for Reconstructing Refractive and Reflective Objects](https://mlsanity.com/paper/2505.05848/) (2025)
- [N-SfC: Robust and Fast Shape Estimation from Caustic Images](https://mlsanity.com/paper/2112.06705/) (2021)
- [Tabletop Transparent Scene Reconstruction via Epipolar-Guided Optical Flow with Monocular Depth Completion Prior](https://mlsanity.com/paper/2310.09956/) (2023)
- [EvaSurf: Efficient View-Aware Implicit Textured Surface Reconstruction](https://mlsanity.com/paper/2311.09806/) (2023)
- [DiffTrans: Differentiable Geometry-Materials Decomposition for Reconstructing Transparent Objects](https://mlsanity.com/paper/2603.00413/) (2026)

## Abstract (arXiv)

Capturing the 3D geometry of transparent objects is a challenging task, ill-suited for general-purpose scanning and reconstruction techniques, since these cannot handle specular light transport phenomena. Existing state-of-the-art methods, designed specifically for this task, either involve a complex setup to reconstruct complete refractive ray paths, or leverage a data-driven approach based on synthetic training data. In either case, the reconstructed 3D models suffer from over-smoothing and loss of fine detail. This paper introduces a novel, high precision, 3D acquisition and reconstruction method for solid transparent objects. Using a static background with a coded pattern, we establish a mapping between the camera view rays and locations on the background. Differentiable tracing of refractive ray paths is then used to directly optimize a 3D mesh approximation of the object, while simultaneously ensuring silhouette consistency and smoothness. Extensive experiments and comparisons demonstrate the superior accuracy of our method.