# One-to-many Reconstruction of 3D Geometry of cultural Artifacts using a synthetically trained Generative Model

arXiv: https://arxiv.org/abs/2402.08310 · HTML: https://mlsanity.com/paper/2402.08310/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 20 references that are in the map.
- First posted on arXiv 13 Feb 2024.

## Builds on (most cited first, 17 of 17)

- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [Pretraining is All You Need for Image-to-Image Translation](https://mlsanity.com/paper/2205.12952/) (2022)
- [Rethinking Image Inpainting via a Mutual Encoder-Decoder with Feature Equalizations](https://mlsanity.com/paper/2007.06929/) (2020)
- [Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Images](https://mlsanity.com/paper/2006.12250/) (2020)
- [Neural 3D Reconstruction in the Wild](https://mlsanity.com/paper/2205.12955/) (2022)
- [3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks](https://mlsanity.com/paper/1707.06375/) (2017)
- [Learning single-image 3D reconstruction by generative modelling of shape, pose and shading](https://mlsanity.com/paper/1901.06447/) (2019)
- [MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpainting](https://mlsanity.com/paper/2203.06304/) (2022)
- [RPM-Net: Recurrent Prediction of Motion and Parts from Point Cloud](https://mlsanity.com/paper/2006.14865/) (2020)
- [Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image](https://mlsanity.com/paper/1707.04682/) (2017)
- [Style-transfer GANs for bridging the domain gap in synthetic pose estimator training](https://mlsanity.com/paper/2004.13681/) (2020)
- [Learning Manifold Patch-Based Representations of Man-Made Shapes](https://mlsanity.com/paper/1906.12337/) (2019)
- [Self-Supervised 2D Image to 3D Shape Translation with Disentangled Representations](https://mlsanity.com/paper/2003.10016/) (2020)
- [A Concept for Reconstructing Stucco Statues from historic Sketches using synthetic Data only](https://mlsanity.com/paper/2402.05593/) (2024)

## Abstract (arXiv)

Estimating the 3D shape of an object using a single image is a difficult problem. Modern approaches achieve good results for general objects, based on real photographs, but worse results on less expressive representations such as historic sketches. Our automated approach generates a variety of detailed 3D representation from a single sketch, depicting a medieval statue, and can be guided by multi-modal inputs, such as text prompts. It relies solely on synthetic data for training, making it adoptable even in cases of only small numbers of training examples. Our solution allows domain experts such as a curators to interactively reconstruct potential appearances of lost artifacts.