# Pose Estimation and 3D Reconstruction of Vehicles from Stereo-Images Using a Subcategory-Aware Shape Prior

arXiv: https://arxiv.org/abs/2107.10898 · HTML: https://mlsanity.com/paper/2107.10898/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 70 references that are in the map.
- First posted on arXiv 22 Jul 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [Real-Time Seamless Single Shot 6D Object Pose Prediction](https://mlsanity.com/paper/1711.08848/) (2017)
- [Stereo R-CNN based 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1902.09738/) (2019)
- [A Large-Scale Car Dataset for Fine-Grained Categorization and Verification](https://mlsanity.com/paper/1506.08959/) (2015)
- [ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape](https://mlsanity.com/paper/1812.02781/) (2018)
- [Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction](https://mlsanity.com/paper/1904.01690/) (2019)
- [Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular image](https://mlsanity.com/paper/1703.07570/) (2017)
- [6-DoF Object Pose from Semantic Keypoints](https://mlsanity.com/paper/1703.04670/) (2017)
- [Category-Specific Object Reconstruction from a Single Image](https://mlsanity.com/paper/1411.6069/) (2014)
- [3D Pose Estimation and 3D Model Retrieval for Objects in the Wild](https://mlsanity.com/paper/1803.11493/) (2018)
- [PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data](https://mlsanity.com/paper/2005.00673/) (2020)
- [DOPS: Learning to Detect 3D Objects and Predict their 3D Shapes](https://mlsanity.com/paper/2004.01170/) (2020)
- [Rethinking Reprojection: Closing the Loop for Pose-aware ShapeReconstruction from a Single Image](https://mlsanity.com/paper/1707.04682/) (2017)
- [Subcategory-aware Convolutional Neural Networks for Object Proposals and Detection](https://mlsanity.com/paper/1604.04693/) (2016)
- [Vehicle Pose and Shape Estimation through Multiple Monocular Vision](https://mlsanity.com/paper/1802.03515/) (2018)
- [Reconstructing Vechicles from a Single Image: Shape Priors for Road Scene Understanding](https://mlsanity.com/paper/1609.09468/) (2016)
- [The Earth ain't Flat: Monocular Reconstruction of Vehicles on Steep and Graded Roads from a Moving Camera](https://mlsanity.com/paper/1803.02057/) (2018)
- [Towards Scene Understanding with Detailed 3D Object Representations](https://mlsanity.com/paper/1411.5935/) (2014)
- [Probabilistic Vehicle Reconstruction Using a Multi-Task CNN](https://mlsanity.com/paper/2102.10681/) (2021)

## Built on by (most cited first, 1 of 1)

- [Online Adaptation for Implicit Object Tracking and Shape Reconstruction in the Wild](https://mlsanity.com/paper/2111.12728/) (2021)

## Abstract (arXiv)

The 3D reconstruction of objects is a prerequisite for many highly relevant applications of computer vision such as mobile robotics or autonomous driving. To deal with the inverse problem of reconstructing 3D objects from their 2D projections, a common strategy is to incorporate prior object knowledge into the reconstruction approach by establishing a 3D model and aligning it to the 2D image plane. However, current approaches are limited due to inadequate shape priors and the insufficiency of the derived image observations for a reliable alignment with the 3D model. The goal of this paper is to show how 3D object reconstruction can profit from a more sophisticated shape prior and from a combined incorporation of different observation types inferred from the images. We introduce a subcategory-aware deformable vehicle model that makes use of a prediction of the vehicle type for a more appropriate regularisation of the vehicle shape. A multi-branch CNN is presented to derive predictions of the vehicle type and orientation. This information is also introduced as prior information for model fitting. Furthermore, the CNN extracts vehicle keypoints and wireframes, which are well-suited for model-to-image association and model fitting. The task of pose estimation and reconstruction is addressed by a versatile probabilistic model. Extensive experiments are conducted using two challenging real-world data sets on both of which the benefit of the developed shape prior can be shown. A comparison to state-of-the-art methods for vehicle pose estimation shows that the proposed approach performs on par or better, confirming the suitability of the developed shape prior and probabilistic model for vehicle reconstruction.