mlsanityData to 14 Oct 2026

Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Real

arXiv 2203.11653 ↗First posted 22 Mar 2022cs.ROcited by 17

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

Autonomous Driving requires high levels of coordination and collaboration between agents. Achieving effective coordination in multi-agent systems is a difficult task that remains largely unresolved. Multi-Agent Reinforcement Learning has arisen as a powerful method to accomplish this task because it considers the interaction between agents and also allows for decentralized training -- which makes it highly scalable. However, transferring policies from simulation to the real world is a big challenge, even for single-agent applications. Multi-agent systems add additional complexities to the Sim-to-Real gap due to agent collaboration and environment synchronization. In this paper, we propose a method to transfer multi-agent autonomous driving policies to the real world. For this, we create a multi-agent environment that imitates the dynamics of the Duckietown multi-robot testbed, and train multi-agent policies using the MAPPO algorithm with different levels of domain randomization. We then transfer the trained policies to the Duckietown testbed and compare the use of the MAPPO algorithm against a traditional rule-based method. We show that the rewards of the transferred policies with MAPPO and domain randomization are, on average, 1.85 times superior to the rule-based method. Moreover, we show that different levels of parameter randomization have a substantial impact on the Sim-to-Real gap.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 18 of its 30 references that are in the map. First posted on arXiv 22 Mar 2022.

Builds on

18 of 30 references.

References · 18

  1. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  6. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  7. 2017On a Formal Model of Safe and Scalable Self-driving Carscited by 131
  8. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 131
  9. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  10. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 117
  11. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 95
  12. 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 90
6 more
  1. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  2. 2020Sim2Real Transfer for Reinforcement Learning without Dynamics Randomizationcited by 47
  3. 2019Bi-level Actor-Critic for Multi-agent Coordinationcited by 39
  4. 2020R-MADDPG for Partially Observable Environments and Limited Communicationcited by 36
  5. 2019DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learningcited by 13
  6. 2020Learning to Play Soccer by Reinforcement and Applying Sim-to-Real to Compete in the Real Worldcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 5; 2025: 5; 2026: 3.

17 papers.

Sort:

By area:

  1. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 38
  2. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 32
  3. 2022RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on statescited by 16
  4. 2024Autonomous Driving Small-Scale Cars: A Survey of Recent Developmentcited by 6
  5. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  6. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Reviewcited by 3
  7. 2025Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learningcited by 3
  8. 2023Learning to Control Autonomous Fleets from Observation via Offline Reinforcement Learningcited by 2
  9. 2024Mixed-Reality Digital Twins: Leveraging the Physical and Virtual Worlds for Hybrid Sim2Real Transition of Multi-Agent Reinforcement Learning Policiescited by 1
  10. 2024Behavioural gap assessment of human-vehicle interaction in real and virtual reality-based scenarios in autonomous drivingcited by 1
  11. 2025Multi-Robot Collaboration through Reinforcement Learning and Abstract Simulationcited by 1
  12. 2025JaxRobotarium: Training and Deploying Multi-Robot Policies in 10 Minutescited by 1
5 more
  1. 2026Prism: Spectral Parameter Sharing for Multi-Agent Reinforcement Learningcited by 1
  2. 2023Progression Cognition Reinforcement Learning with Prioritized Experience for Multi-Vehicle Pursuitcited by 0
  3. 2025Structured Diversity Control: A Dual-Level Framework for Group-Aware Multi-Agent Coordinationcited by 0
  4. 2026Cooperative-Competitive Team Play of Real-World Craft Robotscited by 0
  5. 2026Continual Graph Multi-Agent Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Multi-Agent Deep Reinforcement Learning for Cooperative and Competitive Autonomous Vehicles using AutoDRIVE Ecosystem
  2. 2022Discrete Control in Real-World Driving Environments using Deep Reinforcement Learning
  3. 2021Evaluating the Robustness of Deep Reinforcement Learning for Autonomous Policies in a Multi-agent Urban Driving Environment
  4. 2022Imitation Learning for Generalizable Self-driving Policy with Sim-to-real Transfer
  5. 2021Learning Interactive Driving Policies via Data-driven Simulation
  6. 2026Learning to Drive in New Cities Without Human Demonstrations
  7. 2024Human-compatible driving partners through data-regularized self-play reinforcement learning
  8. 2025SPACeR: Self-Play Anchoring with Centralized Reference Models