# Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic

arXiv: https://arxiv.org/abs/2202.00881 · HTML: https://mlsanity.com/paper/2202.00881/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 44 of its 64 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 30 of 44)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control](https://mlsanity.com/paper/1903.04527/) (2019)
- [Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/1703.06182/) (2017)
- [A Review of Cooperative Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1908.03963/) (2019)
- [A Survey of Learning in Multiagent Environments: Dealing with Non-Stationarity](https://mlsanity.com/paper/1707.09183/) (2017)
- [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020)
- [Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving](https://mlsanity.com/paper/1905.02680/) (2019)
- [Deep Multi-agent Reinforcement Learning for Highway On-Ramp Merging in Mixed Traffic](https://mlsanity.com/paper/2105.05701/) (2021)
- [Social Coordination and Altruism in Autonomous Driving](https://mlsanity.com/paper/2107.00200/) (2021)
- [On the Critical Role of Conventions in Adaptive Human-AI Collaboration](https://mlsanity.com/paper/2104.02871/) (2021)
- [Generative Modeling of Multimodal Multi-Human Behavior](https://mlsanity.com/paper/1803.02015/) (2018)
- [Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints](https://mlsanity.com/paper/1904.00231/) (2019)
- [A learning agent that acquires social norms from public sanctions in decentralized multi-agent settings](https://mlsanity.com/paper/2106.09012/) (2021)
- [Autonomous Highway Driving using Deep Reinforcement Learning](https://mlsanity.com/paper/1904.00035/) (2019)
- [CMetric: A Driving Behavior Measure Using Centrality Functions](https://mlsanity.com/paper/2003.04424/) (2020)
- [Cooperative Autonomous Vehicles that Sympathize with Human Drivers](https://mlsanity.com/paper/2107.00898/) (2021)
- [Towards Full Automated Drive in Urban Environments: A Demonstration in GoMentum Station, California](https://mlsanity.com/paper/1705.01187/) (2017)
- [Deep Local Trajectory Replanning and Control for Robot Navigation](https://mlsanity.com/paper/1905.05279/) (2019)
- [Controlling Steering Angle for Cooperative Self-driving Vehicles utilizing CNN and LSTM-based Deep Networks](https://mlsanity.com/paper/1904.04375/) (2019)
- [Cooperative Perception with Deep Reinforcement Learning for Connected Vehicles](https://mlsanity.com/paper/2004.10927/) (2020)
- [Learning How to Dynamically Route Autonomous Vehicles on Shared Roads](https://mlsanity.com/paper/1909.03664/) (2019)
- [Emergent Prosociality in Multi-Agent Games Through Gifting](https://mlsanity.com/paper/2105.06593/) (2021)
- [Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Critic](https://mlsanity.com/paper/2107.05664/) (2021)
- [Approximate Robust Control of Uncertain Dynamical Systems](https://mlsanity.com/paper/1903.00220/) (2019)
- [Spatio-temporal Dynamics of Cellular V2X Communication in Dense Vehicular Networks](https://mlsanity.com/paper/1906.08634/) (2019)
- [A Maneuver-based Urban Driving Dataset and Model for Cooperative Vehicle Applications](https://mlsanity.com/paper/2005.13781/) (2020)

## Built on by (most cited first, 15 of 15)

- [HM-ViT: Hetero-modal Vehicle-to-Vehicle Cooperative perception with vision transformer](https://mlsanity.com/paper/2304.10628/) (2023)
- [Bridging the Domain Gap for Multi-Agent Perception](https://mlsanity.com/paper/2210.08451/) (2022)
- [Fully Autonomous AI Agents Should Not be Developed](https://mlsanity.com/paper/2502.02649/) (2025)
- [Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications](https://mlsanity.com/paper/2408.10215/) (2024)
- [V2XP-ASG: Generating Adversarial Scenes for Vehicle-to-Everything Perception](https://mlsanity.com/paper/2209.13679/) (2022)
- [CoMAL: Collaborative Multi-Agent Large Language Models for Mixed-Autonomy Traffic](https://mlsanity.com/paper/2410.14368/) (2024)
- [FusionPlanner: A Multi-task Motion Planner for Mining Trucks via Multi-sensor Fusion](https://mlsanity.com/paper/2308.06931/) (2023)
- [Prediction-aware and Reinforcement Learning based Altruistic Cooperative Driving](https://mlsanity.com/paper/2211.10585/) (2022)
- [Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Exploration](https://mlsanity.com/paper/2505.05262/) (2025)
- [A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective](https://mlsanity.com/paper/2503.23650/) (2025)
- [A New Perspective On AI Safety Through Control Theory Methodologies](https://mlsanity.com/paper/2506.23703/) (2025)
- [Shared learning of powertrain control policies for vehicle fleets](https://mlsanity.com/paper/2404.17892/) (2024)
- [A Collaborative Safety Shield for Safe and Efficient CAV Lane Changes in Congested On-Ramp Merging](https://mlsanity.com/paper/2602.10007/) (2026)
- [CRRL: A Causality-Based Reinforcement Learning Framework for Autonomous System Recovery](https://mlsanity.com/paper/2607.03177/) (2026)
- [Think When It Matters: Conditional VLM Reasoning for Social Navigation with RL Policies](https://mlsanity.com/paper/2607.10991/) (2026)

## Abstract (arXiv)

Building autonomous vehicles (AVs) is a complex problem, but enabling them to operate in the real world where they will be surrounded by human-driven vehicles (HVs) is extremely challenging. Prior works have shown the possibilities of creating inter-agent cooperation between a group of AVs that follow a social utility. Such altruistic AVs can form alliances and affect the behavior of HVs to achieve socially desirable outcomes. We identify two major challenges in the co-existence of AVs and HVs. First, social preferences and individual traits of a given human driver, e.g., selflessness and aggressiveness are unknown to an AV, and it is almost impossible to infer them in real-time during a short AV-HV interaction. Second, contrary to AVs that are expected to follow a policy, HVs do not necessarily follow a stationary policy and therefore are extremely hard to predict. To alleviate the above-mentioned challenges, we formulate the mixed-autonomy problem as a multi-agent reinforcement learning (MARL) problem and propose a decentralized framework and reward function for training cooperative AVs. Our approach enables AVs to learn the decision-making of HVs implicitly from experience, optimizes for a social utility while prioritizing safety and allowing adaptability; robustifying altruistic AVs to different human behaviors and constraining them to a safe action space. Finally, we investigate the robustness, safety and sensitivity of AVs to various HVs behavioral traits and present the settings in which the AVs can learn cooperative policies that are adaptable to different situations.