# Deep Learning and Control Algorithms of Direct Perception for Autonomous Driving

arXiv: https://arxiv.org/abs/1910.12031 · HTML: https://mlsanity.com/paper/1910.12031/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2019.

## Built on by (most cited first, 11 of 11)

- [End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving](https://mlsanity.com/paper/2102.04738/) (2021)
- [Expert Human-Level Driving in Gran Turismo Sport Using Deep Reinforcement Learning with Image-based Representation](https://mlsanity.com/paper/2111.06449/) (2021)
- [Vision-based Autonomous Driving for Unstructured Environments Using Imitation Learning](https://mlsanity.com/paper/2202.10002/) (2022)
- [Advancing Autonomous Vehicle Intelligence: Deep Learning and Multimodal LLM for Traffic Sign Recognition and Robust Lane Detection](https://mlsanity.com/paper/2503.06313/) (2025)
- [CI-Net: Contextual Information for Joint Semantic Segmentation and Depth Estimation](https://mlsanity.com/paper/2107.13800/) (2021)
- [Efficient Perception, Planning, and Control Algorithm for Vision-Based Automated Vehicles](https://mlsanity.com/paper/2209.07042/) (2022)
- [Multi-task UNet architecture for end-to-end autonomous driving](https://mlsanity.com/paper/2112.08967/) (2021)
- [Advising Autonomous Cars about the Rules of the Road](https://mlsanity.com/paper/2209.14035/) (2022)
- [Beyond One Model Fits All: Ensemble Deep Learning for Autonomous Vehicles](https://mlsanity.com/paper/2312.05759/) (2023)
- [Tube-Based Robust Control Strategy for Vision-Guided Autonomous Vehicles](https://mlsanity.com/paper/2503.18752/) (2025)
- [VLCQ: Post-training quantization for deep neural networks using variable length coding](https://mlsanity.com/paper/2610.15092/) (2026)

## Abstract (arXiv)

Based on the direct perception paradigm of autonomous driving, we investigate and modify the CNNs (convolutional neural networks) AlexNet and GoogLeNet that map an input image to few perception indicators (heading angle, distances to preceding cars, and distance to road centerline) for estimating driving affordances in highway traffic. We also design a controller with these indicators and the short-range sensor information of TORCS (the open racing car simulator) for driving simulated cars to avoid collisions. We collect a set of images from a TORCS camera in various driving scenarios, train these CNNs using the dataset, test them in unseen traffics, and find that they perform better than earlier algorithms and controllers in terms of training efficiency and driving stability. Source code and data are available on our website.