# Efficient and Robust LiDAR-Based End-to-End Navigation

arXiv: https://arxiv.org/abs/2105.09932 · HTML: https://mlsanity.com/paper/2105.09932/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 52 references that are in the map.
- First posted on arXiv 20 May 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [End-to-end Driving via Conditional Imitation Learning](https://mlsanity.com/paper/1710.02410/) (2017)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [AMC: AutoML for Model Compression and Acceleration on Mobile Devices](https://mlsanity.com/paper/1802.03494/) (2018)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Point-Voxel CNN for Efficient 3D Deep Learning](https://mlsanity.com/paper/1907.03739/) (2019)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentation](https://mlsanity.com/paper/2003.14032/) (2020)
- [Learning to Drive in a Day](https://mlsanity.com/paper/1807.00412/) (2018)
- [SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving](https://mlsanity.com/paper/2003.03653/) (2020)
- [End-to-end Learning of Driving Models from Large-scale Video Datasets](https://mlsanity.com/paper/1612.01079/) (2016)
- [Multimodal End-to-End Autonomous Driving](https://mlsanity.com/paper/1906.03199/) (2019)
- [Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving](https://mlsanity.com/paper/1808.05819/) (2018)
- [Urban Driving with Conditional Imitation Learning](https://mlsanity.com/paper/1912.00177/) (2019)
- [Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control](https://mlsanity.com/paper/1811.06817/) (2018)
- [Octree guided CNN with Spherical Kernels for 3D Point Clouds](https://mlsanity.com/paper/1903.00343/) (2019)
- [Variational End-to-End Navigation and Localization](https://mlsanity.com/paper/1811.10119/) (2018)

## Built on by (most cited first, 22 of 22)

- [VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles](https://mlsanity.com/paper/2111.12083/) (2021)
- [FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer](https://mlsanity.com/paper/2301.08739/) (2023)
- [Prior and Posterior Networks: A Survey on Evidential Deep Learning Methods For Uncertainty Estimation](https://mlsanity.com/paper/2110.03051/) (2021)
- [The Unreasonable Effectiveness of Deep Evidential Regression](https://mlsanity.com/paper/2205.10060/) (2022)
- [TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs](https://mlsanity.com/paper/2311.12862/) (2023)
- [Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?](https://mlsanity.com/paper/2402.09056/) (2024)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [A Comprehensive Survey on Evidential Deep Learning and Its Applications](https://mlsanity.com/paper/2409.04720/) (2024)
- [Perspective-Invariant 3D Object Detection](https://mlsanity.com/paper/2507.17665/) (2025)
- [Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement Learning](https://mlsanity.com/paper/2109.08473/) (2021)
- [Robust Monocular Localization in Sparse HD Maps Leveraging Multi-Task Uncertainty Estimation](https://mlsanity.com/paper/2110.10563/) (2021)
- [NeuPAN: Direct Point Robot Navigation with End-to-End Model-based Learning](https://mlsanity.com/paper/2403.06828/) (2024)
- [DELO: Deep Evidential LiDAR Odometry using Partial Optimal Transport](https://mlsanity.com/paper/2308.07153/) (2023)
- [FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusion](https://mlsanity.com/paper/2308.06931/) (2023)
- [U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequences](https://mlsanity.com/paper/2512.02982/) (2025)
- [Recent Advancements in Deep Learning Applications and Methods for Autonomous Navigation: A Comprehensive Review](https://mlsanity.com/paper/2302.11089/) (2023)
- [TinyLidarNet: 2D LiDAR-based End-to-End Deep Learning Model for F1TENTH Autonomous Racing](https://mlsanity.com/paper/2410.07447/) (2024)
- [LiDAR View Synthesis for Robust Vehicle Navigation Without Expert Labels](https://mlsanity.com/paper/2308.01424/) (2023)
- [V2X-Lead: LiDAR-based End-to-End Autonomous Driving with Vehicle-to-Everything Communication Integration](https://mlsanity.com/paper/2309.15252/) (2023)
- [Solving Vision Tasks with Simple Photoreceptors Instead of Cameras](https://mlsanity.com/paper/2406.11769/) (2024)
- [Joint Localization and Planning using Diffusion](https://mlsanity.com/paper/2409.17995/) (2024)
- [Inter-LPCM: Learning-based Inter-Frame Predictive Coding for LiDAR Point Cloud Compression](https://mlsanity.com/paper/2605.18006/) (2026)

## Abstract (arXiv)

Deep learning has been used to demonstrate end-to-end neural network learning for autonomous vehicle control from raw sensory input. While LiDAR sensors provide reliably accurate information, existing end-to-end driving solutions are mainly based on cameras since processing 3D data requires a large memory footprint and computation cost. On the other hand, increasing the robustness of these systems is also critical; however, even estimating the model's uncertainty is very challenging due to the cost of sampling-based methods. In this paper, we present an efficient and robust LiDAR-based end-to-end navigation framework. We first introduce Fast-LiDARNet that is based on sparse convolution kernel optimization and hardware-aware model design. We then propose Hybrid Evidential Fusion that directly estimates the uncertainty of the prediction from only a single forward pass and then fuses the control predictions intelligently. We evaluate our system on a full-scale vehicle and demonstrate lane-stable as well as navigation capabilities. In the presence of out-of-distribution events (e.g., sensor failures), our system significantly improves robustness and reduces the number of takeovers in the real world.