# Geo-Context Aware Study of Vision-Based Autonomous Driving Models and Spatial Video Data

arXiv: https://arxiv.org/abs/2109.10895 · HTML: https://mlsanity.com/paper/2109.10895/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 65 references that are in the map.
- First posted on arXiv 20 Aug 2021.

## Builds on (most cited first, 25 of 25)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Deep Reinforcement Learning framework for Autonomous Driving](https://mlsanity.com/paper/1704.02532/) (2017)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers](https://mlsanity.com/paper/1801.06889/) (2018)
- [Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car](https://mlsanity.com/paper/1704.07911/) (2017)
- [RetainVis: Visual Analytics with Interpretable and Interactive Recurrent Neural Networks on Electronic Medical Records](https://mlsanity.com/paper/1805.10724/) (2018)
- [End-to-End Race Driving with Deep Reinforcement Learning](https://mlsanity.com/paper/1807.02371/) (2018)
- [Explaining Predictions of Non-Linear Classifiers in NLP](https://mlsanity.com/paper/1606.07298/) (2016)
- [Learning a Driving Simulator](https://mlsanity.com/paper/1608.01230/) (2016)
- [VATLD: A Visual Analytics System to Assess, Understand and Improve Traffic Light Detection](https://mlsanity.com/paper/2009.12975/) (2020)
- [End-to-End Learning of Driving Models with Surround-View Cameras and Route Planners](https://mlsanity.com/paper/1803.10158/) (2018)
- [Interpretable Deep Neural Networks for Single-Trial EEG Classification](https://mlsanity.com/paper/1604.08201/) (2016)
- [A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks](https://mlsanity.com/paper/1606.07757/) (2016)
- [RNNbow: Visualizing Learning via Backpropagation Gradients in Recurrent Neural Networks](https://mlsanity.com/paper/1907.12545/) (2019)

## Abstract (arXiv)

Vision-based deep learning (DL) methods have made great progress in learning autonomous driving models from large-scale crowd-sourced video datasets. They are trained to predict instantaneous driving behaviors from video data captured by on-vehicle cameras. In this paper, we develop a geo-context aware visualization system for the study of Autonomous Driving Model (ADM) predictions together with large-scale ADM video data. The visual study is seamlessly integrated with the geographical environment by combining DL model performance with geospatial visualization techniques. Model performance measures can be studied together with a set of geospatial attributes over map views. Users can also discover and compare prediction behaviors of multiple DL models in both city-wide and street-level analysis, together with road images and video contents. Therefore, the system provides a new visual exploration platform for DL model designers in autonomous driving. Use cases and domain expert evaluation show the utility and effectiveness of the visualization system.