# Tree-structured Policy Planning with Learned Behavior Models

arXiv: https://arxiv.org/abs/2301.11902 · HTML: https://mlsanity.com/paper/2301.11902/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2023.

## Built on by (most cited first, 24 of 24)

- [Parting with Misconceptions about Learning-based Vehicle Motion Planning](https://mlsanity.com/paper/2306.07962/) (2023)
- [LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning](https://mlsanity.com/paper/2401.00125/) (2023)
- [DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Driving](https://mlsanity.com/paper/2310.05885/) (2023)
- [Solving Motion Planning Tasks with a Scalable Generative Model](https://mlsanity.com/paper/2407.02797/) (2024)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [trajdata: A Unified Interface to Multiple Human Trajectory Datasets](https://mlsanity.com/paper/2307.13924/) (2023)
- [Interactive Joint Planning for Autonomous Vehicles](https://mlsanity.com/paper/2310.18301/) (2023)
- [End-To-End Planning of Autonomous Driving in Industry and Academia: 2022-2023](https://mlsanity.com/paper/2401.08658/) (2023)
- [Categorical Traffic Transformer: Interpretable and Diverse Behavior Prediction with Tokenized Latent](https://mlsanity.com/paper/2311.18307/) (2023)
- [Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectives](https://mlsanity.com/paper/2505.09074/) (2025)
- [A Study on Training and Developing Large Language Models for Behavior Tree Generation](https://mlsanity.com/paper/2401.08089/) (2024)
- [Pioneering SE(2)-Equivariant Trajectory Planning for Automated Driving](https://mlsanity.com/paper/2403.11304/) (2024)
- [CoPlanner: An Interactive Motion Planner with Contingency-Aware Diffusion for Autonomous Driving](https://mlsanity.com/paper/2509.17080/) (2025)
- [Perfect Prediction or Plenty of Proposals? What Matters Most in Planning for Autonomous Driving](https://mlsanity.com/paper/2510.15505/) (2025)
- [Multi-Predictor Fusion: Combining Learning-based and Rule-based Trajectory Predictors](https://mlsanity.com/paper/2307.01408/) (2023)
- [HYPE: Hybrid Planning with Ego Proposal-Conditioned Predictions](https://mlsanity.com/paper/2510.12733/) (2025)
- [DECODE: Domain-aware Continual Domain Expansion for Motion Prediction](https://mlsanity.com/paper/2411.17917/) (2024)
- [DiffuSearch: How Hybrid Trajectory Planning Benefits from Aligned Objectives in Diffusion and Action Space](https://mlsanity.com/paper/2609.02252/) (2026)
- [Planning-Aware Diffusion Networks for Enhanced Motion Forecasting in Autonomous Driving](https://mlsanity.com/paper/2410.19639/) (2024)
- [Conditional Prediction by Simulation for Automated Driving](https://mlsanity.com/paper/2502.03286/) (2025)
- [Mode Collapse Happens: Evaluating Critical Interactions in Joint Trajectory Prediction Models](https://mlsanity.com/paper/2506.23164/) (2025)
- [Hope for the Best, Prepare for the Worst: Occlusion-Aware Contingency Planning for Autonomous Vehicles](https://mlsanity.com/paper/2607.03155/) (2026)
- [Optimizing ML Workload Partitioning between CPUs and CIM Accelerators for Heterogeneous Computing](https://mlsanity.com/paper/2607.05240/) (2026)
- [INTERACT: Interactive Planning for Autonomous Driving via Anchor-Conditioned Prediction and Trust-Region Refinement](https://mlsanity.com/paper/2609.31137/) (2026)

## Abstract (arXiv)

Autonomous vehicles (AVs) need to reason about the multimodal behavior of neighboring agents while planning their own motion. Many existing trajectory planners seek a single trajectory that performs well under all plausible futures simultaneously, ignoring bi-directional interactions and thus leading to overly conservative plans. Policy planning, whereby the ego agent plans a policy that reacts to the environment's multimodal behavior, is a promising direction as it can account for the action-reaction interactions between the AV and the environment. However, most existing policy planners do not scale to the complexity of real autonomous vehicle applications: they are either not compatible with modern deep learning prediction models, not interpretable, or not able to generate high quality trajectories. To fill this gap, we propose Tree Policy Planning (TPP), a policy planner that is compatible with state-of-the-art deep learning prediction models, generates multistage motion plans, and accounts for the influence of ego agent on the environment behavior. The key idea of TPP is to reduce the continuous optimization problem into a tractable discrete Markov Decision Process (MDP) through the construction of two tree structures: an ego trajectory tree for ego trajectory options, and a scenario tree for multi-modal ego-conditioned environment predictions. We demonstrate the efficacy of TPP in closed-loop simulations based on real-world nuScenes dataset and results show that TPP scales to realistic AV scenarios and significantly outperforms non-policy baselines.