# Prediction-aware and Reinforcement Learning based Altruistic Cooperative Driving

arXiv: https://arxiv.org/abs/2211.10585 · HTML: https://mlsanity.com/paper/2211.10585/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 36 of its 81 references that are in the map.
- First posted on arXiv 19 Nov 2022.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 30 of 36)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/1703.06182/) (2017)
- [Learning to be Safe: Deep RL with a Safety Critic](https://mlsanity.com/paper/2010.14603/) (2020)
- [Learning to Delegate for Large-scale Vehicle Routing](https://mlsanity.com/paper/2107.04139/) (2021)
- [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020)
- [Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic](https://mlsanity.com/paper/2105.05701/) (2021)
- [Efficient Model Learning for Human-Robot Collaborative Tasks](https://mlsanity.com/paper/1405.6341/) (2014)
- [Social Coordination and Altruism in Autonomous Driving](https://mlsanity.com/paper/2107.00200/) (2021)
- [Generative Modeling of Multimodal Multi-Human Behavior](https://mlsanity.com/paper/1803.02015/) (2018)
- [Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints](https://mlsanity.com/paper/1904.00231/) (2019)
- [Autonomous Highway Driving using Deep Reinforcement Learning](https://mlsanity.com/paper/1904.00035/) (2019)
- [Injecting Planning-Awareness into Prediction and Detection Evaluation](https://mlsanity.com/paper/2110.03270/) (2021)
- [CMetric: A Driving Behavior Measure Using Centrality Functions](https://mlsanity.com/paper/2003.04424/) (2020)
- [Cooperative Autonomous Vehicles that Sympathize with Human Drivers](https://mlsanity.com/paper/2107.00898/) (2021)
- [Towards Full Automated Drive in Urban Environments: A Demonstration in GoMentum Station, California](https://mlsanity.com/paper/1705.01187/) (2017)
- [Deep Local Trajectory Replanning and Control for Robot Navigation](https://mlsanity.com/paper/1905.05279/) (2019)
- [Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic](https://mlsanity.com/paper/2202.00881/) (2022)
- [Cooperative Perception with Deep Reinforcement Learning for Connected Vehicles](https://mlsanity.com/paper/2004.10927/) (2020)
- [Learning How to Dynamically Route Autonomous Vehicles on Shared Roads](https://mlsanity.com/paper/1909.03664/) (2019)
- [Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic](https://mlsanity.com/paper/2107.05664/) (2021)
- [Approximate Robust Control of Uncertain Dynamical Systems](https://mlsanity.com/paper/1903.00220/) (2019)
- [A Taxonomy and Review of Algorithms for Modeling and Predicting Human Driver Behavior](https://mlsanity.com/paper/2006.08832/) (2020)
- [Spatio-temporal Dynamics of Cellular V2X Communication in Dense Vehicular Networks](https://mlsanity.com/paper/1906.08634/) (2019)
- [Real-Time Hardware-In-the-Loop Emulation Framework for DSRC-based Connected Vehicle Applications](https://mlsanity.com/paper/1905.09267/) (2019)
- [V2X System Architecture Utilizing Hybrid Gaussian Process-based Model Structures](https://mlsanity.com/paper/1903.01576/) (2019)
- [Gaussian Process based Stochastic Model Predictive Control for Cooperative Adaptive Cruise Control](https://mlsanity.com/paper/2111.07162/) (2021)
- [Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving](https://mlsanity.com/paper/2105.06517/) (2021)
- [Impact of Information Flow Topology on Safety of Tightly-coupled Connected and Automated Vehicle Platoons Utilizing Stochastic Control](https://mlsanity.com/paper/2203.15772/) (2022)
- [Representing Realistic Human Driver Behaviors using a Finite Size Gaussian Process Kernel Bank](https://mlsanity.com/paper/2003.13837/) (2020)

## Built on by (most cited first, 6 of 6)

- [A Safe and Efficient Self-evolving Algorithm for Decision-making and Control of Autonomous Driving Systems](https://mlsanity.com/paper/2408.12187/) (2024)
- [Context-Aware Target Classification with Hybrid Gaussian Process prediction for Cooperative Vehicle Safety systems](https://mlsanity.com/paper/2212.12819/) (2022)
- [Human-Like Implicit Intention Expression for Autonomous Driving Motion Planning: A Method Based on Learning Human Intention Priors](https://mlsanity.com/paper/2307.15950/) (2023)
- [Intrinsic Dynamics-Driven Generalizable Scene Representations for Vision-Oriented Decision-Making Applications](https://mlsanity.com/paper/2405.19736/) (2024)
- [OPTIMA: Optimized Policy for Intelligent Multi-Agent Systems Enables Coordination-Aware Autonomous Vehicles](https://mlsanity.com/paper/2410.18112/) (2024)
- [A Reliable Context-Aware and Temporal Planning Framework for Autonomous Driving](https://mlsanity.com/paper/2607.04689/) (2026)

## Abstract (arXiv)

Autonomous vehicle (AV) navigation in the presence of Human-driven vehicles (HVs) is challenging, as HVs continuously update their policies in response to AVs. In order to navigate safely in the presence of complex AV-HV social interactions, the AVs must learn to predict these changes. Humans are capable of navigating such challenging social interaction settings because of their intrinsic knowledge about other agents behaviors and use that to forecast what might happen in the future. Inspired by humans, we provide our AVs the capability of anticipating future states and leveraging prediction in a cooperative reinforcement learning (RL) decision-making framework, to improve safety and robustness. In this paper, we propose an integration of two essential and earlier-presented components of AVs: social navigation and prediction. We formulate the AV decision-making process as a RL problem and seek to obtain optimal policies that produce socially beneficial results utilizing a prediction-aware planning and social-aware optimization RL framework. We also propose a Hybrid Predictive Network (HPN) that anticipates future observations. The HPN is used in a multi-step prediction chain to compute a window of predicted future observations to be used by the value function network (VFN). Finally, a safe VFN is trained to optimize a social utility using a sequence of previous and predicted observations, and a safety prioritizer is used to leverage the interpretable kinematic predictions to mask the unsafe actions, constraining the RL policy. We compare our prediction-aware AV to state-of-the-art solutions and demonstrate performance improvements in terms of efficiency and safety in multiple simulated scenarios.