# Probabilistic End-to-End Vehicle Navigation in Complex Dynamic Environments with Multimodal Sensor Fusion

arXiv: https://arxiv.org/abs/2005.01935 · HTML: https://mlsanity.com/paper/2005.01935/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2020.

## Built on by (most cited first, 24 of 24)

- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation](https://mlsanity.com/paper/2104.12861/) (2021)
- [Vision-Based Autonomous Car Racing Using Deep Imitative Reinforcement Learning](https://mlsanity.com/paper/2107.08325/) (2021)
- [End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Driving](https://mlsanity.com/paper/2104.08862/) (2021)
- [DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networks](https://mlsanity.com/paper/2108.05030/) (2021)
- [Hierarchical Generative Adversarial Imitation Learning with Mid-level Input Generation for Autonomous Driving on Urban Environments](https://mlsanity.com/paper/2302.04823/) (2023)
- [Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement Learning](https://mlsanity.com/paper/2109.08473/) (2021)
- [Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectory](https://mlsanity.com/paper/2010.10393/) (2020)
- [RLAD: Reinforcement Learning from Pixels for Autonomous Driving in Urban Environments](https://mlsanity.com/paper/2305.18510/) (2023)
- [Autonomous Navigation through intersections with Graph ConvolutionalNetworks and Conditional Imitation Learning for Self-driving Cars](https://mlsanity.com/paper/2102.00675/) (2021)
- [Self-Supervised Domain Adaptation for Visual Navigation with Global Map Consistency](https://mlsanity.com/paper/2110.07184/) (2021)
- [Conquering Ghosts: Relation Learning for Information Reliability Representation and End-to-End Robust Navigation](https://mlsanity.com/paper/2203.09952/) (2022)
- [Graph-based Prediction and Planning Policy Network (GP3Net) for scalable self-driving in dynamic environments using Deep Reinforcement Learning](https://mlsanity.com/paper/2312.05784/) (2023)
- [Revolutionizing Battery Disassembly: The Design and Implementation of a Battery Disassembly Autonomous Mobile Manipulator Robot(BEAM-1)](https://mlsanity.com/paper/2407.06590/) (2024)
- [Using Ensemble Diffusion to Estimate Uncertainty for End-to-End Autonomous Driving](https://mlsanity.com/paper/2506.00560/) (2025)
- [A survey on deep learning approaches for data integration in autonomous driving system](https://mlsanity.com/paper/2306.11740/) (2023)
- [CARIL: Confidence-Aware Regression in Imitation Learning for Autonomous Driving](https://mlsanity.com/paper/2503.00783/) (2025)
- [Collision Risk Estimation via Loss Prediction in End-to-End Autonomous Driving](https://mlsanity.com/paper/2503.07425/) (2025)
- [Distributionally Robust Imitation Learning: Layered Control Architecture for Certifiable Autonomy](https://mlsanity.com/paper/2512.17899/) (2025)
- [GreenEyes: An Air Quality Evaluating Model based on WaveNet](https://mlsanity.com/paper/2212.04175/) (2022)
- [Predictive Maneuver Planning with Deep Reinforcement Learning (PMP-DRL) for comfortable and safe autonomous driving](https://mlsanity.com/paper/2306.09055/) (2023)
- [Imitation Learning for Autonomous Driving: Insights from Real-World Testing](https://mlsanity.com/paper/2504.18847/) (2025)
- [Investigating Behavior Cloning from Few Demonstrations for Autonomous Driving based on Bird's-Eye View in Simulated Cities](https://mlsanity.com/paper/2610.20203/) (2026)

## Abstract (arXiv)

All-day and all-weather navigation is a critical capability for autonomous driving, which requires proper reaction to varied environmental conditions and complex agent behaviors. Recently, with the rise of deep learning, end-to-end control for autonomous vehicles has been well studied. However, most works are solely based on visual information, which can be degraded by challenging illumination conditions such as dim light or total darkness. In addition, they usually generate and apply deterministic control commands without considering the uncertainties in the future. In this paper, based on imitation learning, we propose a probabilistic driving model with ultiperception capability utilizing the information from the camera, lidar and radar. We further evaluate its driving performance online on our new driving benchmark, which includes various environmental conditions (e.g., urban and rural areas, traffic densities, weather and times of the day) and dynamic obstacles (e.g., vehicles, pedestrians, motorcyclists and bicyclists). The results suggest that our proposed model outperforms baselines and achieves excellent generalization performance in unseen environments with heavy traffic and extreme weather.