# Disentangling and Vectorization: A 3D Visual Perception Approach for Autonomous Driving Based on Surround-View Fisheye Cameras

arXiv: https://arxiv.org/abs/2107.08862 · HTML: https://mlsanity.com/paper/2107.08862/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 19 Jul 2021.

## Builds on (most cited first, 23 of 23)

- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [3D Bounding Box Estimation Using Deep Learning and Geometry](https://mlsanity.com/paper/1612.00496/) (2016)
- [M3D-RPN: Monocular 3D Region Proposal Network for Object Detection](https://mlsanity.com/paper/1907.06038/) (2019)
- [SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimation](https://mlsanity.com/paper/2002.10111/) (2020)
- [WoodScape: A multi-task, multi-camera fisheye dataset for autonomous driving](https://mlsanity.com/paper/1905.01489/) (2019)
- [Accurate Monocular Object Detection via Color-Embedded 3D Reconstruction for Autonomous Driving](https://mlsanity.com/paper/1903.11444/) (2019)
- [Vehicle Detection from 3D Lidar Using Fully Convolutional Network](https://mlsanity.com/paper/1608.07916/) (2016)
- [Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction](https://mlsanity.com/paper/1904.01690/) (2019)
- [Deep MANTA: A Coarse-to-fine Many-Task Network for joint 2D and 3D vehicle analysis from monocular image](https://mlsanity.com/paper/1703.07570/) (2017)
- [MonoGRNet: A Geometric Reasoning Network for Monocular 3D Object Localization](https://mlsanity.com/paper/1811.10247/) (2018)
- [Ground-aware Monocular 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2102.00690/) (2021)
- [OmniDet: Surround View Cameras based Multi-task Visual Perception Network for Autonomous Driving](https://mlsanity.com/paper/2102.07448/) (2021)
- [SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2008.04017/) (2020)
- [BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud](https://mlsanity.com/paper/2006.11436/) (2020)
- [FisheyeDistanceNet: Self-Supervised Scale-Aware Distance Estimation using Monocular Fisheye Camera for Autonomous Driving](https://mlsanity.com/paper/1910.04076/) (2019)
- [Eliminating the Blind Spot: Adapting 3D Object Detection and Monocular Depth Estimation to 360deg Panoramic Imagery](https://mlsanity.com/paper/1808.06253/) (2018)
- [FisheyeMODNet: Moving Object detection on Surround-view Cameras for Autonomous Driving](https://mlsanity.com/paper/1908.11789/) (2019)
- [End to End Vehicle Lateral Control Using a Single Fisheye Camera](https://mlsanity.com/paper/1808.06940/) (2018)
- [Accurate, Low-Latency Visual Perception for Autonomous Racing:Challenges, Mechanisms, and Practical Solutions](https://mlsanity.com/paper/2007.13971/) (2020)
- [Real-time Detection, Tracking, and Classification of Moving and Stationary Objects using Multiple Fisheye Images](https://mlsanity.com/paper/1803.06077/) (2018)
- [FisheyeMultiNet: Real-time Multi-task Learning Architecture for Surround-view Automated Parking System](https://mlsanity.com/paper/1912.11066/) (2019)
- [Vehicle Re-ID for Surround-view Camera System](https://mlsanity.com/paper/2006.16503/) (2020)

## Built on by (most cited first, 4 of 4)

- [Surround-view Fisheye BEV-Perception for Valet Parking: Dataset, Baseline and Distortion-insensitive Multi-task Framework](https://mlsanity.com/paper/2212.04111/) (2022)
- [Rethinking Camera Choice: An Empirical Study on Fisheye Camera Properties in Robotic Manipulation](https://mlsanity.com/paper/2603.02139/) (2026)
- [LineMarkNet: Line Landmark Detection for Valet Parking](https://mlsanity.com/paper/2309.10475/) (2023)
- [PPD: A New Valet Parking Pedestrian Fisheye Dataset for Autonomous Driving](https://mlsanity.com/paper/2309.11002/) (2023)

## Abstract (arXiv)

The 3D visual perception for vehicles with the surround-view fisheye camera system is a critical and challenging task for low-cost urban autonomous driving. While existing monocular 3D object detection methods perform not well enough on the fisheye images for mass production, partly due to the lack of 3D datasets of such images. In this paper, we manage to overcome and avoid the difficulty of acquiring the large scale of accurate 3D labeled truth data, by breaking down the 3D object detection task into some sub-tasks, such as vehicle's contact point detection, type classification, re-identification and unit assembling, etc. Particularly, we propose the concept of Multidimensional Vector to include the utilizable information generated in different dimensions and stages, instead of the descriptive approach for the bird's eye view (BEV) or a cube of eight points. The experiments of real fisheye images demonstrate that our solution achieves state-of-the-art accuracy while being real-time in practice.