# Learning Reward Models for Cooperative Trajectory Planning with Inverse Reinforcement Learning and Monte Carlo Tree Search

arXiv: https://arxiv.org/abs/2202.06443 · HTML: https://mlsanity.com/paper/2202.06443/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 8 of its 16 references that are in the map.
- First posted on arXiv 14 Feb 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Builds on (most cited first, 8 of 8)

- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Efficient Sampling-Based Maximum Entropy Inverse Reinforcement Learning with Application to Autonomous Driving](https://mlsanity.com/paper/2006.13704/) (2020)
- [Watch This: Scalable Cost-Function Learning for Path Planning in Urban Environments](https://mlsanity.com/paper/1607.02329/) (2016)
- [Adaptive Behavior Generation for Autonomous Driving using Deep Reinforcement Learning with Compact Semantic States](https://mlsanity.com/paper/1809.03214/) (2018)
- [Decentralized Cooperative Planning for Automated Vehicles with Continuous Monte Carlo Tree Search](https://mlsanity.com/paper/1809.03200/) (2018)
- [Generalizing Decision Making for Automated Driving with an Invariant Environment Representation using Deep Reinforcement Learning](https://mlsanity.com/paper/2102.06765/) (2021)

## Built on by (most cited first, 3 of 3)

- [Holistic Deep-Reinforcement-Learning-based Training of Autonomous Navigation Systems](https://mlsanity.com/paper/2302.02921/) (2023)
- [BIDA: A Bi-level Interaction Decision-making Algorithm for Autonomous Vehicles in Dynamic Traffic Scenarios](https://mlsanity.com/paper/2506.16546/) (2025)
- [State-of-the-art in Robot Learning for Multi-Robot Collaboration: A Comprehensive Survey](https://mlsanity.com/paper/2408.11822/) (2024)

## Abstract (arXiv)

Cooperative trajectory planning methods for automated vehicles can solve traffic scenarios that require a high degree of cooperation between traffic participants. However, for cooperative systems to integrate into human-centered traffic, the automated systems must behave human-like so that humans can anticipate the system's decisions. While Reinforcement Learning has made remarkable progress in solving the decision-making part, it is non-trivial to parameterize a reward model that yields predictable actions. This work employs feature-based Maximum Entropy Inverse Reinforcement Learning combined with Monte Carlo Tree Search to learn reward models that maximize the likelihood of recorded multi-agent cooperative expert trajectories. The evaluation demonstrates that the approach can recover a reasonable reward model that mimics the expert and performs similarly to a manually tuned baseline reward model.