# Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image

arXiv: https://arxiv.org/abs/1707.04682 · HTML: https://mlsanity.com/paper/1707.04682/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2017.

## Built on by (most cited first, 20 of 20)

- [Neural Wavelet-domain Diffusion for 3D Shape Generation](https://mlsanity.com/paper/2209.08725/) (2022)
- [Motion Inspired Unsupervised Perception and Prediction in Autonomous Driving](https://mlsanity.com/paper/2210.08061/) (2022)
- [Learning Generative Models of Textured 3D Meshes from Real-World Images](https://mlsanity.com/paper/2103.15627/) (2021)
- [Neural Template: Topology-aware Reconstruction and Disentangled Generation of 3D Meshes](https://mlsanity.com/paper/2206.04942/) (2022)
- [Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion](https://mlsanity.com/paper/2211.11674/) (2022)
- [HVH: Learning a Hybrid Neural Volumetric Representation for Dynamic Hair Performance Capture](https://mlsanity.com/paper/2112.06904/) (2021)
- [Neural Wavelet-domain Diffusion for 3D Shape Generation, Inversion, and Manipulation](https://mlsanity.com/paper/2302.00190/) (2023)
- [Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolation](https://mlsanity.com/paper/2205.05922/) (2022)
- [PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learning](https://mlsanity.com/paper/2105.05643/) (2021)
- [Self-Supervised Point Cloud Completion via Inpainting](https://mlsanity.com/paper/2111.10701/) (2021)
- [Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization](https://mlsanity.com/paper/2305.03043/) (2023)
- [Structural Causal 3D Reconstruction](https://mlsanity.com/paper/2207.10156/) (2022)
- [SDFEst: Categorical Pose and Shape Estimation of Objects from RGB-D using Signed Distance Fields](https://mlsanity.com/paper/2207.04880/) (2022)
- [Multi-Category Mesh Reconstruction From Image Collections](https://mlsanity.com/paper/2110.11256/) (2021)
- [Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images](https://mlsanity.com/paper/2109.02288/) (2021)
- [Learning Compositional Shape Priors for Few-Shot 3D Reconstruction](https://mlsanity.com/paper/2106.06440/) (2021)
- [A View Independent Classification Framework for Yoga Postures](https://mlsanity.com/paper/2206.13577/) (2022)
- [Pose Estimation and 3D Reconstruction of Vehicles from Stereo-Images Using a Subcategory-Aware Shape Prior](https://mlsanity.com/paper/2107.10898/) (2021)
- [Two-stage Synthetic Supervising and Multi-view Consistency Self-supervising based Animal 3D Reconstruction by Single Image](https://mlsanity.com/paper/2311.13199/) (2023)
- [One-to-many Reconstruction of 3D Geometry of cultural Artifacts using a synthetically trained Generative Model](https://mlsanity.com/paper/2402.08310/) (2024)

## Abstract (arXiv)

An emerging problem in computer vision is the reconstruction of 3D shape and pose of an object from a single image. Hitherto, the problem has been addressed through the application of canonical deep learning methods to regress from the image directly to the 3D shape and pose labels. These approaches, however, are problematic from two perspectives. First, they are minimizing the error between 3D shapes and pose labels - with little thought about the nature of this label error when reprojecting the shape back onto the image. Second, they rely on the onerous and ill-posed task of hand labeling natural images with respect to 3D shape and pose. In this paper we define the new task of pose-aware shape reconstruction from a single image, and we advocate that cheaper 2D annotations of objects silhouettes in natural images can be utilized. We design architectures of pose-aware shape reconstruction which re-project the predicted shape back on to the image using the predicted pose. Our evaluation on several object categories demonstrates the superiority of our method for predicting pose-aware 3D shapes from natural images.