# Predictive Control for Autonomous Driving with Uncertain, Multi-modal Predictions

arXiv: https://arxiv.org/abs/2310.20561 · HTML: https://mlsanity.com/paper/2310.20561/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 24 of its 55 references that are in the map.
- First posted on arXiv 31 Oct 2023.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 24 of 24)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving](https://mlsanity.com/paper/2310.05885/) (2023)
- [Interactive multi-modal motion planning with Branch Model Predictive Control](https://mlsanity.com/paper/2109.05128/) (2021)
- [Autonomous Driving Strategies at Intersections: Scenarios, State-of-the-Art, and Future Outlooks](https://mlsanity.com/paper/2106.13052/) (2021)
- [Stochastic Model Predictive Control with a Safety Guarantee for Automated Driving: Extended Version](https://mlsanity.com/paper/2009.09381/) (2020)
- [Interaction-Aware Trajectory Prediction and Planning for Autonomous Vehicles in Forced Merge Scenarios](https://mlsanity.com/paper/2112.07624/) (2021)
- [MARC: Multipolicy and Risk-aware Contingency Planning for Autonomous Driving](https://mlsanity.com/paper/2308.12021/) (2023)
- [Interaction-Aware Motion Planning for Autonomous Vehicles with Multi-Modal Obstacle Uncertainty Predictions](https://mlsanity.com/paper/2212.11819/) (2022)
- [Scenario-Based Trajectory Optimization in Uncertain Dynamic Environments](https://mlsanity.com/paper/2103.12517/) (2021)
- [Gaussian Process-based Stochastic Model Predictive Control for Overtaking in Autonomous Racing](https://mlsanity.com/paper/2105.12236/) (2021)
- [Stochastic MPC with Multi-modal Predictions for Traffic Intersections](https://mlsanity.com/paper/2109.09792/) (2021)
- [Moving Obstacle Avoidance: a Data-Driven Risk-Aware Approach](https://mlsanity.com/paper/2203.14913/) (2022)
- [Models and Predictive Control for Nonplanar Vehicle Navigation](https://mlsanity.com/paper/2104.08427/) (2021)
- [Safe, Learning-Based MPC for Highway Driving under Lane-Change Uncertainty: A Distributionally Robust Approach](https://mlsanity.com/paper/2206.13319/) (2022)
- [Collision Avoidance for Dynamic Obstacles with Uncertain Predictions using Model Predictive Control](https://mlsanity.com/paper/2208.03529/) (2022)
- [Convexified Open-Loop Stochastic Optimal Control for Linear Non-Gaussian Systems](https://mlsanity.com/paper/2010.02101/) (2020)
- [Stochastic MPC with Dual Control for Autonomous Driving with Multi-Modal Interaction-Aware Predictions](https://mlsanity.com/paper/2208.03525/) (2022)
- [Energy-Efficient Driving in Connected Corridors via Minimum Principle Control: Vehicle-in-the-Loop Experimental Verification in Mixed Fleets](https://mlsanity.com/paper/2211.09658/) (2022)
- [Interaction and Decision Making-aware Motion Planning using Branch Model Predictive Control](https://mlsanity.com/paper/2302.00060/) (2023)
- [Energy-Efficient Lane Changes Planning and Control for Connected Autonomous Vehicles on Urban Roads](https://mlsanity.com/paper/2304.08576/) (2023)

## Built on by (most cited first, 4 of 4)

- [Scalable Multi-modal Model Predictive Control via Duality-based Interaction Predictions](https://mlsanity.com/paper/2402.01116/) (2024)
- [Tuning the feedback controller gains is a simple way to improve autonomous driving performance](https://mlsanity.com/paper/2402.05064/) (2024)
- [Safe and Nonconservative Contingency Planning for Autonomous Vehicles via Online Learning-Based Reachable Set Barriers](https://mlsanity.com/paper/2509.07464/) (2025)
- [Closed-Loop Refinement and Execution for Learned Driving Planners](https://mlsanity.com/paper/2610.00992/) (2026)

## Abstract (arXiv)

We propose a Stochastic MPC (SMPC) formulation for path planning with autonomous vehicles in scenarios involving multiple agents with multi-modal predictions. The multi-modal predictions capture the uncertainty of urban driving in distinct modes/maneuvers (e.g., yield, keep speed) and driving trajectories (e.g., speed, turning radius), which are incorporated for multi-modal collision avoidance chance constraints for path planning. In the presence of multi-modal uncertainties, it is challenging to reliably compute feasible path planning solutions at real-time frequencies (≥ 10 Hz). Our main technological contribution is a convex SMPC formulation that simultaneously (1) optimizes over parameterized feedback policies and (2) allocates risk levels for each mode of the prediction. The use of feedback policies and risk allocation enhances the feasibility and performance of the SMPC formulation against multi-modal predictions with large uncertainty. We evaluate our approach via simulations and road experiments with a full-scale vehicle interacting in closed-loop with virtual vehicles. We consider distinct, multi-modal driving scenarios: 1) Negotiating a traffic light and a fast, tailgating agent, 2) Executing an unprotected left turn at a traffic intersection, and 3) Changing lanes in the presence of multiple agents. For all of these scenarios, our approach reliably computes multi-modal solutions to the path-planning problem at real-time frequencies.