# Multimodal End-to-End Learning for Autonomous Steering in Adverse Road and Weather Conditions

arXiv: https://arxiv.org/abs/2010.14924 · HTML: https://mlsanity.com/paper/2010.14924/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2020.

## Built on by (most cited first, 5 of 5)

- [Multi-Modal Fusion for Sensorimotor Coordination in Steering Angle Prediction](https://mlsanity.com/paper/2202.05500/) (2022)
- [Dense Road Surface Grip Map Prediction from Multimodal Image Data](https://mlsanity.com/paper/2404.17324/) (2024)
- [Optical Flow Matters: an Empirical Comparative Study on Fusing Monocular Extracted Modalities for Better Steering](https://mlsanity.com/paper/2409.12716/) (2024)
- [Leveraging Road Area Semantic Segmentation with Auxiliary Steering Task](https://mlsanity.com/paper/2212.09438/) (2022)
- [Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy](https://mlsanity.com/paper/2511.11777/) (2025)

## Abstract (arXiv)

Autonomous driving is challenging in adverse road and weather conditions in which there might not be lane lines, the road might be covered in snow and the visibility might be poor. We extend the previous work on end-to-end learning for autonomous steering to operate in these adverse real-life conditions with multimodal data. We collected 28 hours of driving data in several road and weather conditions and trained convolutional neural networks to predict the car steering wheel angle from front-facing color camera images and lidar range and reflectance data. We compared the CNN model performances based on the different modalities and our results show that the lidar modality improves the performances of different multimodal sensor-fusion models. We also performed on-road tests with different models and they support this observation.