NeuroSMPC: A Neural Network guided Sampling Based MPC for On-Road Autonomous Driving
On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners
Abstract (arXiv)
In this paper we show an effective means of integrating data driven frameworks to sampling based optimal control to vastly reduce the compute time for easy adoption and adaptation to real time applications such as on-road autonomous driving in the presence of dynamic actors. Presented with training examples, a spatio-temporal CNN learns to predict the optimal mean control over a finite horizon that precludes further resampling, an iterative process that makes sampling based optimal control formulations difficult to adopt in real time settings. Generating control samples around the network-predicted optimal mean retains the advantage of sample diversity while enabling real time rollout of trajectories that avoids multiple dynamic obstacles in an on-road navigation setting. Further the 3D CNN architecture implicitly learns the future trajectories of the dynamic agents in the scene resulting in successful collision free navigation despite no explicit future trajectory prediction. We show performance gain over multiple baselines in a number of on-road scenes through closed loop simulations in CARLA. We also showcase the real world applicability of our system by running it on our custom Autonomous Driving Platform (AutoDP).
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 16 of its 28 references that are in the map. First posted on arXiv 19 Oct 2023.
Builds on
16 of 28 references.
References · 16
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
- 2022Language-driven Semantic Segmentationcited by 711
- 2023Symbolic Discovery of Optimization Algorithmscited by 423
- 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
- 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 302
- 2021End-to-end Interpretable Neural Motion Plannercited by 240
- 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
- 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
- 2015Model Predictive Path Integral Control using Covariance Variable Importance Samplingcited by 122
4 more
- 2020Sample-efficient Cross-Entropy Method for Real-time Planningcited by 90
- 2020MonoLayout: Amodal scene layout from a single imagecited by 49
- 2021STORM: An Integrated Framework for Fast Joint-Space Model-Predictive Control for Reactive Manipulationcited by 46
- 2020Model-Predictive Control via Cross-Entropy and Gradient-Based Optimizationcited by 32
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Learning to Sample: Reinforcement Learning-Guided Sampling for Autonomous Vehicle Motion Planning
- 2023End-to-End Learning of Behavioural Inputs for Autonomous Driving in Dense Traffic
- 2023Model Predictive Inferential Control of Neural State-Space Models for Autonomous Vehicle Motion Planning ↗
- 2023Learning the References of Online Model Predictive Control for Urban Self-Driving
- 2025Inverse RL Scene Dynamics Learning for Nonlinear Predictive Control in Autonomous Vehicles
- 2021ObserveNet Control: A Vision-Dynamics Learning Approach to Predictive Control in Autonomous Vehicles
- 2025A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approach
- 2025Imitation Learning for Autonomous Driving: Insights from Real-World Testing