# Improving 3D Object Detection for Pedestrians with Virtual Multi-View Synthesis Orientation Estimation

arXiv: https://arxiv.org/abs/1907.06777 · HTML: https://mlsanity.com/paper/1907.06777/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2019.

## Built on by (most cited first, 8 of 8)

- [Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion](https://mlsanity.com/paper/2203.09780/) (2022)
- [Multi-modal Sensor Fusion for Auto Driving Perception: A Survey](https://mlsanity.com/paper/2202.02703/) (2022)
- [GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2310.08261/) (2023)
- [HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detection](https://mlsanity.com/paper/2206.15157/) (2022)
- [FGR: Frustum-Aware Geometric Reasoning for Weakly Supervised 3D Vehicle Detection](https://mlsanity.com/paper/2105.07647/) (2021)
- [DRINet: A Dual-Representation Iterative Learning Network for Point Cloud Segmentation](https://mlsanity.com/paper/2108.04023/) (2021)
- [Accurate and Real-time 3D Pedestrian Detection Using an Efficient Attentive Pillar Network](https://mlsanity.com/paper/2112.15458/) (2021)
- [Novel View Synthesis from a Single RGBD Image for Indoor Scenes](https://mlsanity.com/paper/2311.01065/) (2023)

## Abstract (arXiv)

Accurately estimating the orientation of pedestrians is an important and challenging task for autonomous driving because this information is essential for tracking and predicting pedestrian behavior. This paper presents a flexible Virtual Multi-View Synthesis module that can be adopted into 3D object detection methods to improve orientation estimation. The module uses a multi-step process to acquire the fine-grained semantic information required for accurate orientation estimation. First, the scene's point cloud is densified using a structure preserving depth completion algorithm and each point is colorized using its corresponding RGB pixel. Next, virtual cameras are placed around each object in the densified point cloud to generate novel viewpoints, which preserve the object's appearance. We show that this module greatly improves the orientation estimation on the challenging pedestrian class on the KITTI benchmark. When used with the open-source 3D detector AVOD-FPN, we outperform all other published methods on the pedestrian Orientation, 3D, and Bird's Eye View benchmarks.