# End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies

arXiv: https://arxiv.org/abs/1710.03804 · HTML: https://mlsanity.com/paper/1710.03804/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Oct 2017.

## Built on by (most cited first, 16 of 16)

- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [Deep Federated Learning for Autonomous Driving](https://mlsanity.com/paper/2110.05754/) (2021)
- [Federated Learning on the Road: Autonomous Controller Design for Connected and Autonomous Vehicles](https://mlsanity.com/paper/2102.03401/) (2021)
- [Milestones in Autonomous Driving and Intelligent Vehicles Part I: Control, Computing System Design, Communication, HD Map, Testing, and Human Behaviors](https://mlsanity.com/paper/2305.11239/) (2023)
- [Real-time End-to-End Federated Learning: An Automotive Case Study](https://mlsanity.com/paper/2103.11879/) (2021)
- [Multi-modal Fusion Technology based on Vehicle Information: A Survey](https://mlsanity.com/paper/2211.06080/) (2022)
- [LiDAR-as-Camera for End-to-End Driving](https://mlsanity.com/paper/2206.15170/) (2022)
- [Prospective Role of Foundation Models in Advancing Autonomous Vehicles](https://mlsanity.com/paper/2405.02288/) (2023)
- [Dynamic Conditional Imitation Learning for Autonomous Driving](https://mlsanity.com/paper/2211.11579/) (2022)
- [Improving Generalization of Transfer Learning Across Domains Using Spatio-Temporal Features in Autonomous Driving](https://mlsanity.com/paper/2103.08116/) (2021)
- [Controlling Steering with Energy-Based Models](https://mlsanity.com/paper/2301.12264/) (2023)
- [Optical Flow Matters: an Empirical Comparative Study on Fusing Monocular Extracted Modalities for Better Steering](https://mlsanity.com/paper/2409.12716/) (2024)
- [Human Insights Driven Latent Space for Different Driving Perspectives: A Unified Encoder for Efficient Multi-Task Inference](https://mlsanity.com/paper/2409.10095/) (2024)
- [A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approach](https://mlsanity.com/paper/2512.11944/) (2025)
- [Memory based neural networks for end-to-end autonomous driving](https://mlsanity.com/paper/2205.12124/) (2022)
- [Enhancing Steering Estimation with Semantic-Aware GNNs](https://mlsanity.com/paper/2503.17153/) (2025)

## Abstract (arXiv)

Steering a car through traffic is a complex task that is difficult to cast into algorithms. Therefore, researchers turn to training artificial neural networks from front-facing camera data stream along with the associated steering angles. Nevertheless, most existing solutions consider only the visual camera frames as input, thus ignoring the temporal relationship between frames. In this work, we propose a Convolutional Long Short-Term Memory Recurrent Neural Network (C-LSTM), that is end-to-end trainable, to learn both visual and dynamic temporal dependencies of driving. Additionally, We introduce posing the steering angle regression problem as classification while imposing a spatial relationship between the output layer neurons. Such method is based on learning a sinusoidal function that encodes steering angles. To train and validate our proposed methods, we used the publicly available Comma.ai dataset. Our solution improved steering root mean square error by 35% over recent methods, and led to a more stable steering by 87%.