# Deep Optimized Priors for 3D Shape Modeling and Reconstruction

arXiv: https://arxiv.org/abs/2012.07241 · HTML: https://mlsanity.com/paper/2012.07241/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 20 of 20)

- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks](https://mlsanity.com/paper/2105.03582/) (2021)
- [Surface Reconstruction from Point Clouds by Learning Predictive Context Priors](https://mlsanity.com/paper/2204.11015/) (2022)
- [Surface Reconstruction from Point Clouds: A Survey and a Benchmark](https://mlsanity.com/paper/2205.02413/) (2022)
- [Task-Generic Hierarchical Human Motion Prior using VAEs](https://mlsanity.com/paper/2106.04004/) (2021)
- [Learning Parallel Dense Correspondence from Spatio-Temporal Descriptors for Efficient and Robust 4D Reconstruction](https://mlsanity.com/paper/2103.16341/) (2021)
- [Representing 3D Shapes with Probabilistic Directed Distance Fields](https://mlsanity.com/paper/2112.05300/) (2021)
- [Mending Neural Implicit Modeling for 3D Vehicle Reconstruction in the Wild](https://mlsanity.com/paper/2101.06860/) (2021)
- [FvOR: Robust Joint Shape and Pose Optimization for Few-view Object Reconstruction](https://mlsanity.com/paper/2205.07763/) (2022)
- [Reconstructing Interacting Hands with Interaction Prior from Monocular Images](https://mlsanity.com/paper/2308.14082/) (2023)
- [DeepMend: Learning Occupancy Functions to Represent Shape for Repair](https://mlsanity.com/paper/2210.05728/) (2022)
- [DeepJoin: Learning a Joint Occupancy, Signed Distance, and Normal Field Function for Shape Repair](https://mlsanity.com/paper/2211.12400/) (2022)
- [Micrometer: Micromechanics Transformer for Predicting Mechanical Responses of Heterogeneous Materials](https://mlsanity.com/paper/2410.05281/) (2024)
- [HM3D-ABO: A Photo-realistic Dataset for Object-centric Multi-view 3D Reconstruction](https://mlsanity.com/paper/2206.12356/) (2022)
- [3D Reconstruction of Sculptures from Single Images via Unsupervised Domain Adaptation on Implicit Models](https://mlsanity.com/paper/2210.04265/) (2022)
- [Equivariant Light Field Convolution and Transformer](https://mlsanity.com/paper/2212.14871/) (2022)
- [Isomorphic mesh generation from point clouds with multilayer perceptrons](https://mlsanity.com/paper/2210.14157/) (2022)
- [Edge Preserving Implicit Surface Representation of Point Clouds](https://mlsanity.com/paper/2301.04860/) (2023)
- [Development of a deep learning platform for optimising sheet stamping geometries subject to manufacturing constraints](https://mlsanity.com/paper/2202.03422/) (2022)
- [Probabilistic Directed Distance Fields for Ray-Based Shape Representations](https://mlsanity.com/paper/2404.09081/) (2024)

## Abstract (arXiv)

Many learning-based approaches have difficulty scaling to unseen data, as the generality of its learned prior is limited to the scale and variations of the training samples. This holds particularly true with 3D learning tasks, given the sparsity of 3D datasets available. We introduce a new learning framework for 3D modeling and reconstruction that greatly improves the generalization ability of a deep generator. Our approach strives to connect the good ends of both learning-based and optimization-based methods. In particular, unlike the common practice that fixes the pre-trained priors at test time, we propose to further optimize the learned prior and latent code according to the input physical measurements after the training. We show that the proposed strategy effectively breaks the barriers constrained by the pre-trained priors and could lead to high-quality adaptation to unseen data. We realize our framework using the implicit surface representation and validate the efficacy of our approach in a variety of challenging tasks that take highly sparse or collapsed observations as input. Experimental results show that our approach compares favorably with the state-of-the-art methods in terms of both generality and accuracy.