# Photometric Mesh Optimization for Video-Aligned 3D Object Reconstruction

arXiv: https://arxiv.org/abs/1903.08642 · HTML: https://mlsanity.com/paper/1903.08642/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Mar 2019.

## Built on by (most cited first, 28 of 28)

- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [CodeNeRF: Disentangled Neural Radiance Fields for Object Categories](https://mlsanity.com/paper/2109.01750/) (2021)
- [POCO: Point Convolution for Surface Reconstruction](https://mlsanity.com/paper/2201.01831/) (2022)
- [PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction](https://mlsanity.com/paper/2302.10668/) (2023)
- [Local-to-Global Registration for Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2211.11505/) (2022)
- [GIFS: Neural Implicit Function for General Shape Representation](https://mlsanity.com/paper/2204.07126/) (2022)
- [Multi-View Mesh Reconstruction with Neural Deferred Shading](https://mlsanity.com/paper/2212.04386/) (2022)
- [DSP-SLAM: Object Oriented SLAM with Deep Shape Priors](https://mlsanity.com/paper/2108.09481/) (2021)
- [Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolation](https://mlsanity.com/paper/2205.05922/) (2022)
- [FvOR: Robust Joint Shape and Pose Optimization for Few-view Object Reconstruction](https://mlsanity.com/paper/2205.07763/) (2022)
- [Neural Vector Fields: Implicit Representation by Explicit Learning](https://mlsanity.com/paper/2303.04341/) (2023)
- [H4D: Human 4D Modeling by Learning Neural Compositional Representation](https://mlsanity.com/paper/2203.01247/) (2022)
- [Deep Feedback Inverse Problem Solver](https://mlsanity.com/paper/2101.07719/) (2021)
- [EvAC3D: From Event-based Apparent Contours to 3D Models via Continuous Visual Hulls](https://mlsanity.com/paper/2304.05296/) (2023)
- [FIRe: Fast Inverse Rendering using Directional and Signed Distance Functions](https://mlsanity.com/paper/2203.16284/) (2022)
- [LightSAL: Lightweight Sign Agnostic Learning for Implicit Surface Representation](https://mlsanity.com/paper/2103.14273/) (2021)
- [Multi-Category Mesh Reconstruction From Image Collections](https://mlsanity.com/paper/2110.11256/) (2021)
- [Dynamic Convolution for 3D Point Cloud Instance Segmentation](https://mlsanity.com/paper/2107.08392/) (2021)
- [Black-Box Test-Time Shape REFINEment for Single View 3D Reconstruction](https://mlsanity.com/paper/2108.09911/) (2021)
- [A Review of Deep Learning-Powered Mesh Reconstruction Methods](https://mlsanity.com/paper/2303.02879/) (2023)
- [Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization](https://mlsanity.com/paper/2309.01512/) (2023)
- [Self-Supervised Dual Contouring](https://mlsanity.com/paper/2405.18131/) (2024)
- [From None to All: Self-Supervised 3D Reconstruction via Novel View Synthesis](https://mlsanity.com/paper/2603.27455/) (2026)
- [Self-supervised learning of object pose estimation using keypoint prediction](https://mlsanity.com/paper/2302.07360/) (2023)
- [KRONC: Keypoint-based Robust Camera Optimization for 3D Car Reconstruction](https://mlsanity.com/paper/2409.05407/) (2024)
- [Convolutional Neural Network-based Efficient Dense Point Cloud Generation using Unsigned Distance Fields](https://mlsanity.com/paper/2203.11537/) (2022)
- [EvaSurf: Efficient View-Aware Implicit Textured Surface Reconstruction](https://mlsanity.com/paper/2311.09806/) (2023)
- [FA-BARF: Frequency Adapted Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2503.12086/) (2025)

## Abstract (arXiv)

In this paper, we address the problem of 3D object mesh reconstruction from RGB videos. Our approach combines the best of multi-view geometric and data-driven methods for 3D reconstruction by optimizing object meshes for multi-view photometric consistency while constraining mesh deformations with a shape prior. We pose this as a piecewise image alignment problem for each mesh face projection. Our approach allows us to update shape parameters from the photometric error without any depth or mask information. Moreover, we show how to avoid a degeneracy of zero photometric gradients via rasterizing from a virtual viewpoint. We demonstrate 3D object mesh reconstruction results from both synthetic and real-world videos with our photometric mesh optimization, which is unachievable with either na"ive mesh generation networks or traditional pipelines of surface reconstruction without heavy manual post-processing.