mlsanityData to 7 Oct 2026

MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning

arXiv 2109.12674 ↗First posted 26 Sep 2021cs.LG cs.ROcited by 281

On the map: World models → World models for autonomous driving

also in 1 more
Abstract (arXiv)

Driving safely requires multiple capabilities from human and intelligent agents, such as the generalizability to unseen environments, the safety awareness of the surrounding traffic, and the decision-making in complex multi-agent settings. Despite the great success of Reinforcement Learning (RL), most of the RL research works investigate each capability separately due to the lack of integrated environments. In this work, we develop a new driving simulation platform called MetaDrive to support the research of generalizable reinforcement learning algorithms for machine autonomy. MetaDrive is highly compositional, which can generate an infinite number of diverse driving scenarios from both the procedural generation and the real data importing. Based on MetaDrive, we construct a variety of RL tasks and baselines in both single-agent and multi-agent settings, including benchmarking generalizability across unseen scenes, safe exploration, and learning multi-agent traffic. The generalization experiments conducted on both procedurally generated scenarios and real-world scenarios show that increasing the diversity and the size of the training set leads to the improvement of the RL agent's generalizability. We further evaluate various safe reinforcement learning and multi-agent reinforcement learning algorithms in MetaDrive environments and provide the benchmarks. Source code, documentation, and demo video are available at url https://metadriverse.github.io/metadrive.

Key facts. Cited by 281 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 34 of its 69 references that are in the map. First posted on arXiv 26 Sep 2021.

Builds on

34 of 69 references.

Closest ancestors · 1

Same direction, newest first

  1. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  7. 2017Constrained Policy Optimizationcited by 928
  8. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  9. 2018Learning Dexterous In-Hand Manipulationcited by 643
  10. 2019The StarCraft Multi-Agent Challengecited by 589
  11. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  12. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
21 more
  1. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 350
  2. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  3. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  4. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 281
  5. 2018Learning to Drive in a Daycited by 266
  6. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  7. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  8. 2021iGibson 2.0: Object-Centric Simulation for Robot Learning of Everyday Household Taskscited by 191
  9. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  10. 2019CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenariocited by 114
  11. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 103
  12. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  13. 2019SUMMIT: A Simulator for Urban Driving in Massive Mixed Trafficcited by 39
  14. 2017Sim4CV: A Photo-Realistic Simulator for Computer Vision Applicationscited by 31
  15. 2017Beyond Grand Theft Auto V for Training, Testing and Enhancing Deep Learning in Self Driving Carscited by 26
  16. 2019Open-Sourced Reinforcement Learning Environments for Surgical Roboticscited by 24
  17. 2019A Review of Reinforcement Learning for Autonomous Building Energy Managementcited by 18
  18. 2019Learning a Decision Module by Imitating Driver's Control Behaviorscited by 17
  19. 2020MADRaS : Multi Agent Driving Simulatorcited by 15
  20. 2018Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learningcited by 13
  21. 2020Emergent Road Rules In Multi-Agent Driving Environmentscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 15; 2023: 33; 2024: 62; 2025: 93; 2026: 74.

281 papers.

Sort:

By area:

  1. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  2. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  3. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  4. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  5. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  6. 2023Waymax: An Accelerated, Data-Driven Simulator for Large-Scale Autonomous Driving Researchcited by 158
  7. 2025Pseudo-Simulation for Autonomous Drivingcited by 141
  8. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 135
  9. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  10. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  11. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  12. 2023Street-View Image Generation from a Bird's-Eye View Layoutcited by 102
38 more
  1. 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
  2. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 93
  3. 2023FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Conditioncited by 89
  4. 2023The Waymo Open Sim Agents Challengecited by 86
  5. 2023Language Conditioned Traffic Generationcited by 86
  6. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 80
  7. 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 76
  8. 2022SafeBench: A Benchmarking Platform for Safety Evaluation of Autonomous Vehiclescited by 65
  9. 2025Robust Autonomy Emerges from Self-Playcited by 64
  10. 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Modelcited by 62
  11. 2025A Survey of World Models for Autonomous Drivingcited by 60
  12. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 58
  13. 2022Nocturne: a scalable driving benchmark for bringing multi-agent learning one step closer to the real worldcited by 56
  14. 2023How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligencecited by 55
  15. 2023CAT: Closed-loop Adversarial Training for Safe End-to-End Drivingcited by 52
  16. 2024UniTraj: A Unified Framework for Scalable Vehicle Trajectory Predictioncited by 51
  17. 2023SafeDreamer: Safe Reinforcement Learning with World Modelscited by 50
  18. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 49
  19. 2022Efficient Learning of Safe Driving Policy via Human-AI Copilot Optimizationcited by 44
  20. 2024GPUDrive: Data-driven, multi-agent driving simulation at 1 million FPScited by 43
  21. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environmentcited by 42
  22. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
  23. 2024GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstructioncited by 39
  24. 2025RAP: 3D Rasterization Augmented End-to-End Planningcited by 39
  25. 2025Foundation Models in Autonomous Driving: A Survey on Scenario Generation and Scenario Analysiscited by 37
  26. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priorscited by 36
  27. 2024SimGen: Simulator-conditioned Driving Scene Generationcited by 34
  28. 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
  29. 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 34
  30. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  31. 2024Ctrl-X: Controlling Structure and Appearance for Text-To-Image Generation Without Guidancecited by 33
  32. 2024HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Drivingcited by 31
  33. 2022Behavioral Intention Prediction in Driving Scenes: A Surveycited by 30
  34. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  35. 2025Vid2Sim: Realistic and Interactive Simulation from Video for Urban Navigationcited by 30
  36. 2021Safe Driving via Expert Guided Policy Optimizationcited by 29
  37. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  38. 2025The Role of World Models in Shaping Autonomous Driving: A Comprehensive Surveycited by 29

50 most cited of 281 All 281 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Meta-Multi-Agent Reinforcement Learning for Fast Adaptation of Interactive Policies with Applications to Autonomous Driving
  2. 2026MTDrive: Multi-turn Interactive Reinforcement Learning for Autonomous Driving
  3. 2020Meta Reinforcement Learning-Based Lane Change Strategy for Autonomous Vehicles
  4. 2025Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning
  5. 2021Towards Learning Generalizable Driving Policies from Restricted Latent Representations
  6. 2023Multi-Agent Deep Reinforcement Learning for Cooperative and Competitive Autonomous Vehicles using AutoDRIVE Ecosystem
  7. 2026Scaling Curriculum Learning For Autonomous Driving
  8. 2025Diverse and Adaptive Behavior Curriculum for Autonomous Driving: A Student-Teacher Framework with Multi-Agent RL