# High-Quality RGB-D Reconstruction via Multi-View Uncalibrated Photometric Stereo and Gradient-SDF

arXiv: https://arxiv.org/abs/2210.12202 · HTML: https://mlsanity.com/paper/2210.12202/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 48 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 17 of 17)

- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Neural RGB-D Surface Reconstruction](https://mlsanity.com/paper/2104.04532/) (2021)
- [Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline](https://mlsanity.com/paper/2104.06174/) (2021)
- [Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting](https://mlsanity.com/paper/1708.01670/) (2017)
- [A Differential Volumetric Approach to Multi-View Photometric Stereo](https://mlsanity.com/paper/1811.01984/) (2018)
- [Variational Uncalibrated Photometric Stereo under General Lighting](https://mlsanity.com/paper/1904.03942/) (2019)
- [LED-based Photometric Stereo: Modeling, Calibration and Numerical Solution](https://mlsanity.com/paper/1707.01018/) (2017)
- [Gradient-SDF: A Semi-Implicit Surface Representation for 3D Reconstruction](https://mlsanity.com/paper/2111.13652/) (2021)
- [Inferring Super-Resolution Depth from a Moving Light-Source Enhanced RGB-D Sensor: A Variational Approach](https://mlsanity.com/paper/1912.06501/) (2019)
- [Depth Super-Resolution Meets Uncalibrated Photometric Stereo](https://mlsanity.com/paper/1708.00411/) (2017)
- [On the well-posedness of uncalibrated photometric stereo under general lighting](https://mlsanity.com/paper/1911.07268/) (2019)

## Built on by (most cited first, 5 of 5)

- [4Deform: Neural Surface Deformation for Robust Shape Interpolation](https://mlsanity.com/paper/2502.20208/) (2025)
- [Sparse Views, Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo](https://mlsanity.com/paper/2404.00098/) (2024)
- [SimpleMapping: Real-Time Visual-Inertial Dense Mapping with Deep Multi-View Stereo](https://mlsanity.com/paper/2306.08648/) (2023)
- [Enhancing Surface Neural Implicits with Curvature-Guided Sampling and Uncertainty-Augmented Representations](https://mlsanity.com/paper/2306.02099/) (2023)
- [Coloring the Past: Neural Historical Buildings Reconstruction from Archival Photography](https://mlsanity.com/paper/2311.17810/) (2023)

## Abstract (arXiv)

Fine-detailed reconstructions are in high demand in many applications. However, most of the existing RGB-D reconstruction methods rely on pre-calculated accurate camera poses to recover the detailed surface geometry, where the representation of a surface needs to be adapted when optimizing different quantities. In this paper, we present a novel multi-view RGB-D based reconstruction method that tackles camera pose, lighting, albedo, and surface normal estimation via the utilization of a gradient signed distance field (gradient-SDF). The proposed method formulates the image rendering process using specific physically-based model(s) and optimizes the surface's quantities on the actual surface using its volumetric representation, as opposed to other works which estimate surface quantities only near the actual surface. To validate our method, we investigate two physically-based image formation models for natural light and point light source applications. The experimental results on synthetic and real-world datasets demonstrate that the proposed method can recover high-quality geometry of the surface more faithfully than the state-of-the-art and further improves the accuracy of estimated camera poses.