# Spectral MVIR: Joint Reconstruction of 3D Shape and Spectral Reflectance

arXiv: https://arxiv.org/abs/2104.07308 · HTML: https://mlsanity.com/paper/2104.07308/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 54 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 3 of 3)

- [Polarimetric Multi-View Inverse Rendering](https://mlsanity.com/paper/2007.08830/) (2020)
- [Pro-Cam SSfM: Projector-Camera System for Structure and Spectral Reflectance from Motion](https://mlsanity.com/paper/1908.08185/) (2019)
- [Spectral Reflectance Estimation Using Projector with Unknown Spectral Power Distribution](https://mlsanity.com/paper/2012.10083/) (2020)

## Built on by (most cited first, 8 of 8)

- [PANDORA: Polarization-Aided Neural Decomposition Of Radiance](https://mlsanity.com/paper/2203.13458/) (2022)
- [NeISF: Neural Incident Stokes Field for Geometry and Material Estimation](https://mlsanity.com/paper/2311.13187/) (2023)
- [Spec-NeRF: Multi-spectral Neural Radiance Fields](https://mlsanity.com/paper/2310.12987/) (2023)
- [Deep Hyperspectral-Depth Reconstruction Using Single Color-Dot Projection](https://mlsanity.com/paper/2204.03929/) (2022)
- [Dispersed Structured Light for Hyperspectral 3D Imaging](https://mlsanity.com/paper/2311.18287/) (2023)
- [Spectral 3D Computer Vision -- A Review](https://mlsanity.com/paper/2302.08054/) (2023)
- [Automatic Spectral Calibration of Hyperspectral Images:Method, Dataset and Benchmark](https://mlsanity.com/paper/2412.14925/) (2024)
- [Polarimetric Multi-View Inverse Rendering](https://mlsanity.com/paper/2212.12721/) (2022)

## Abstract (arXiv)

Reconstructing an object's high-quality 3D shape with inherent spectral reflectance property, beyond typical device-dependent RGB albedos, opens the door to applications requiring a high-fidelity 3D model in terms of both geometry and photometry. In this paper, we propose a novel Multi-View Inverse Rendering (MVIR) method called Spectral MVIR for jointly reconstructing the 3D shape and the spectral reflectance for each point of object surfaces from multi-view images captured using a standard RGB camera and low-cost lighting equipment such as an LED bulb or an LED projector. Our main contributions are twofold: (i) We present a rendering model that considers both geometric and photometric principles in the image formation by explicitly considering camera spectral sensitivity, light's spectral power distribution, and light source positions. (ii) Based on the derived model, we build a cost-optimization MVIR framework for the joint reconstruction of the 3D shape and the per-vertex spectral reflectance while estimating the light source positions and the shadows. Different from most existing spectral-3D acquisition methods, our method does not require expensive special equipment and cumbersome geometric calibration. Experimental results using both synthetic and real-world data demonstrate that our Spectral MVIR can acquire a high-quality 3D model with accurate spectral reflectance property.