# Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictors

arXiv: https://arxiv.org/abs/2307.01408 · HTML: https://mlsanity.com/paper/2307.01408/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 22 of 22)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [What the Constant Velocity Model Can Teach Us About Pedestrian Motion Prediction](https://mlsanity.com/paper/1903.07933/) (2019)
- [ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planning](https://mlsanity.com/paper/2206.13387/) (2022)
- [Learning to Predict Vehicle Trajectories with Model-based Planning](https://mlsanity.com/paper/2103.04027/) (2021)
- [How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics](https://mlsanity.com/paper/1710.11040/) (2017)
- [Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methods](https://mlsanity.com/paper/2008.00097/) (2020)
- [Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction](https://mlsanity.com/paper/1710.09483/) (2017)
- [Liability, Ethics, and Culture-Aware Behavior Specification using Rulebooks](https://mlsanity.com/paper/1902.09355/) (2019)
- [DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles](https://mlsanity.com/paper/2212.06437/) (2022)
- [PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation](https://mlsanity.com/paper/2109.11094/) (2021)
- [Tree-structured Policy Planning with Learned Behavior Models](https://mlsanity.com/paper/2301.11902/) (2023)
- [Rule-based Optimal Control for Autonomous Driving](https://mlsanity.com/paper/2101.05709/) (2021)
- [The Reasonable Crowd: Towards evidence-based and interpretable models of driving behavior](https://mlsanity.com/paper/2107.13507/) (2021)
- [On complementing end-to-end human behavior predictors with planning](https://mlsanity.com/paper/2103.05661/) (2021)
- [Receding Horizon Planning with Rule Hierarchies for Autonomous Vehicles](https://mlsanity.com/paper/2212.03323/) (2022)
- [Planning-inspired Hierarchical Trajectory Prediction for Autonomous Driving](https://mlsanity.com/paper/2304.11295/) (2023)

## Built on by (most cited first, 3 of 3)

- [Driving Everywhere with Large Language Model Policy Adaptation](https://mlsanity.com/paper/2402.05932/) (2024)
- [Adaptive Prediction Ensemble: Improving Out-of-Distribution Generalization of Motion Forecasting](https://mlsanity.com/paper/2407.09475/) (2024)
- [RuleFuser: An Evidential Bayes Approach for Rule Injection in Imitation Learned Planners and Predictors for Robustness under Distribution Shifts](https://mlsanity.com/paper/2405.11139/) (2024)

## Abstract (arXiv)

Trajectory prediction modules are key enablers for safe and efficient planning of autonomous vehicles (AVs), particularly in highly interactive traffic scenarios. Recently, learning-based trajectory predictors have experienced considerable success in providing state-of-the-art performance due to their ability to learn multimodal behaviors of other agents from data. In this paper, we present an algorithm called multi-predictor fusion (MPF) that augments the performance of learning-based predictors by imbuing them with motion planners that are tasked with satisfying logic-based rules. MPF probabilistically combines learning- and rule-based predictors by mixing trajectories from both standalone predictors in accordance with a belief distribution that reflects the online performance of each predictor. In our results, we show that MPF outperforms the two standalone predictors on various metrics and delivers the most consistent performance.