# PerMO: Perceiving More at Once from a Single Image for Autonomous Driving

arXiv: https://arxiv.org/abs/2007.08116 · HTML: https://mlsanity.com/paper/2007.08116/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2020.

## Built on by (most cited first, 5 of 5)

- [AutoShape: Real-Time Shape-Aware Monocular 3D Object Detection](https://mlsanity.com/paper/2108.11127/) (2021)
- [CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulation](https://mlsanity.com/paper/2311.01447/) (2023)
- [Reconstructing Objects in-the-wild for Realistic Sensor Simulation](https://mlsanity.com/paper/2311.05602/) (2023)
- [Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation](https://mlsanity.com/paper/2311.01446/) (2023)
- [Robust 2D/3D Vehicle Parsing in CVIS](https://mlsanity.com/paper/2103.06432/) (2021)

## Abstract (arXiv)

We present a novel approach to detect, segment, and reconstruct complete textured 3D models of vehicles from a single image for autonomous driving. Our approach combines the strengths of deep learning and the elegance of traditional techniques from part-based deformable model representation to produce high-quality 3D models in the presence of severe occlusions. We present a new part-based deformable vehicle model that is used for instance segmentation and automatically generate a dataset that contains dense correspondences between 2D images and 3D models. We also present a novel end-to-end deep neural network to predict dense 2D/3D mapping and highlight its benefits. Based on the dense mapping, we are able to compute precise 6-DoF poses and 3D reconstruction results at almost interactive rates on a commodity GPU. We have integrated these algorithms with an autonomous driving system. In practice, our method outperforms the state-of-the-art methods for all major vehicle parsing tasks: 2D instance segmentation by 4.4 points (mAP), 6-DoF pose estimation by 9.11 points, and 3D detection by 1.37. Moreover, we have released all of the source code, dataset, and the trained model on Github.