# Monocular Plan View Networks for Autonomous Driving

arXiv: https://arxiv.org/abs/1905.06937 · HTML: https://mlsanity.com/paper/1905.06937/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 May 2019.

## Built on by (most cited first, 30 of 37)

- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [Causal Confusion in Imitation Learning](https://mlsanity.com/paper/1905.11979/) (2019)
- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [End-to-End Urban Driving by Imitating a Reinforcement Learning Coach](https://mlsanity.com/paper/2108.08265/) (2021)
- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [Multimodal Virtual Point 3D Detection](https://mlsanity.com/paper/2111.06881/) (2021)
- [DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets](https://mlsanity.com/paper/2301.06051/) (2023)
- [Translating Images into Maps](https://mlsanity.com/paper/2110.00966/) (2021)
- [PlanT: Explainable Planning Transformers via Object-Level Representations](https://mlsanity.com/paper/2210.14222/) (2022)
- [PLUTO: Pushing the Limit of Imitation Learning-based Planning for Autonomous Driving](https://mlsanity.com/paper/2404.14327/) (2024)
- [Virtual Sparse Convolution for Multimodal 3D Object Detection](https://mlsanity.com/paper/2303.02314/) (2023)
- [UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representation](https://mlsanity.com/paper/2308.07732/) (2023)
- [Fighting Copycat Agents in Behavioral Cloning from Observation Histories](https://mlsanity.com/paper/2010.14876/) (2020)
- [CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds](https://mlsanity.com/paper/2210.04264/) (2022)
- [RBGNet: Ray-based Grouping for 3D Object Detection](https://mlsanity.com/paper/2204.02251/) (2022)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [Keyframe-Focused Visual Imitation Learning](https://mlsanity.com/paper/2106.06452/) (2021)
- [Rethinking Imitation-based Planner for Autonomous Driving](https://mlsanity.com/paper/2309.10443/) (2023)
- [Learning by Watching](https://mlsanity.com/paper/2106.05966/) (2021)
- [SelfD: Self-Learning Large-Scale Driving Policies From the Web](https://mlsanity.com/paper/2204.10320/) (2022)
- [Resolving Copycat Problems in Visual Imitation Learning via Residual Action Prediction](https://mlsanity.com/paper/2207.09705/) (2022)
- [BLOS-BEV: Navigation Map Enhanced Lane Segmentation Network, Beyond Line of Sight](https://mlsanity.com/paper/2407.08526/) (2024)
- [Boosting 3D Object Detection via Object-Focused Image Fusion](https://mlsanity.com/paper/2207.10589/) (2022)
- [DROID: Driver-centric Risk Object Identification](https://mlsanity.com/paper/2106.13201/) (2021)
- [Driving Through Ghosts: Behavioral Cloning with False Positives](https://mlsanity.com/paper/2008.12969/) (2020)
- [CarFormer: Self-Driving with Learned Object-Centric Representations](https://mlsanity.com/paper/2407.15843/) (2024)
- [Causal Imitative Model for Autonomous Driving](https://mlsanity.com/paper/2112.03908/) (2021)
- [M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Driving](https://mlsanity.com/paper/2403.12552/) (2024)
- ["The Pedestrian next to the Lamppost" Adaptive Object Graphs for Better Instantaneous Mapping](https://mlsanity.com/paper/2204.02944/) (2022)

## Abstract (arXiv)

Convolutions on monocular dash cam videos capture spatial invariances in the image plane but do not explicitly reason about distances and depth. We propose a simple transformation of observations into a bird's eye view, also known as plan view, for end-to-end control. We detect vehicles and pedestrians in the first person view and project them into an overhead plan view. This representation provides an abstraction of the environment from which a deep network can easily deduce the positions and directions of entities. Additionally, the plan view enables us to leverage advances in 3D object detection in conjunction with deep policy learning. We evaluate our monocular plan view network on the photo-realistic Grand Theft Auto V simulator. A network using both a plan view and front view causes less than half as many collisions as previous detection-based methods and an order of magnitude fewer collisions than pure pixel-based policies.