# Multi-Agent Deep Reinforcement Learning for Cooperative and Competitive Autonomous Vehicles using AutoDRIVE Ecosystem

arXiv: https://arxiv.org/abs/2309.10007 · HTML: https://mlsanity.com/paper/2309.10007/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2001.11231/) (2020)
- [Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning](https://mlsanity.com/paper/1709.10082/) (2017)
- [Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learning](https://mlsanity.com/paper/2008.07971/) (2020)
- [Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing](https://mlsanity.com/paper/2202.07008/) (2022)
- [Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learning](https://mlsanity.com/paper/2103.14666/) (2021)
- [Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning](https://mlsanity.com/paper/2001.06627/) (2020)
- [F1/10: An Open-Source Autonomous Cyber-Physical Platform](https://mlsanity.com/paper/1901.08567/) (2019)
- [AutoDRIVE: A Comprehensive, Flexible and Integrated Digital Twin Ecosystem for Enhancing Autonomous Driving Research and Education](https://mlsanity.com/paper/2212.05241/) (2022)
- [AutoDRIVE Simulator: A Simulator for Scaled Autonomous Vehicle Research and Education](https://mlsanity.com/paper/2103.10030/) (2021)
- [Decentralized Motion Planning for Multi-Robot Navigation using Deep Reinforcement Learning](https://mlsanity.com/paper/2011.05605/) (2020)
- [Towards Sim2Real Transfer of Autonomy Algorithms using AutoDRIVE Ecosystem](https://mlsanity.com/paper/2307.13272/) (2023)
- [AutoDRIVE -- Technical Report](https://mlsanity.com/paper/2211.08475/) (2022)
- [AutoDRIVE Simulator -- Technical Report](https://mlsanity.com/paper/2211.07022/) (2022)
- [Towards Mechatronics Approach of System Design, Verification and Validation for Autonomous Vehicles](https://mlsanity.com/paper/2301.13425/) (2023)
- [Autonomous Racing using a Hybrid Imitation-Reinforcement Learning Architecture](https://mlsanity.com/paper/2110.05437/) (2021)

## Abstract (arXiv)

This work presents a modular and parallelizable multi-agent deep reinforcement learning framework for imbibing cooperative as well as competitive behaviors within autonomous vehicles. We introduce AutoDRIVE Ecosystem as an enabler to develop physically accurate and graphically realistic digital twins of Nigel and F1TENTH, two scaled autonomous vehicle platforms with unique qualities and capabilities, and leverage this ecosystem to train and deploy multi-agent reinforcement learning policies. We first investigate an intersection traversal problem using a set of cooperative vehicles (Nigel) that share limited state information with each other in single as well as multi-agent learning settings using a common policy approach. We then investigate an adversarial head-to-head autonomous racing problem using a different set of vehicles (F1TENTH) in a multi-agent learning setting using an individual policy approach. In either set of experiments, a decentralized learning architecture was adopted, which allowed robust training and testing of the approaches in stochastic environments, since the agents were mutually independent and exhibited asynchronous motion behavior. The problems were further aggravated by providing the agents with sparse observation spaces and requiring them to sample control commands that implicitly satisfied the imposed kinodynamic as well as safety constraints. The experimental results for both problem statements are reported in terms of quantitative metrics and qualitative remarks for training as well as deployment phases.