mlsanityData to 7 Oct 2026

Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving

arXiv 1610.03295 ↗First posted 11 Oct 2016cs.AI cs.LG stat.MLcited by 301

Not on the map. Why?

Cited by 33 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Autonomous driving is a multi-agent setting where the host vehicle must apply sophisticated negotiation skills with other road users when overtaking, giving way, merging, taking left and right turns and while pushing ahead in unstructured urban roadways. Since there are many possible scenarios, manually tackling all possible cases will likely yield a too simplistic policy. Moreover, one must balance between unexpected behavior of other drivers/pedestrians and at the same time not to be too defensive so that normal traffic flow is maintained. In this paper we apply deep reinforcement learning to the problem of forming long term driving strategies. We note that there are two major challenges that make autonomous driving different from other robotic tasks. First, is the necessity for ensuring functional safety - something that machine learning has difficulty with given that performance is optimized at the level of an expectation over many instances. Second, the Markov Decision Process model often used in robotics is problematic in our case because of unpredictable behavior of other agents in this multi-agent scenario. We make three contributions in our work. First, we show how policy gradient iterations can be used without Markovian assumptions. Second, we decompose the problem into a composition of a Policy for Desires (which is to be learned) and trajectory planning with hard constraints (which is not learned). The goal of Desires is to enable comfort of driving, while hard constraints guarantees the safety of driving. Third, we introduce a hierarchical temporal abstraction we call an "Option Graph" with a gating mechanism that significantly reduces the effective horizon and thereby reducing the variance of the gradient estimation even further.

Key facts. Cited by 301 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 2; 2020: 3; 2021: 70; 2022: 50; 2023: 59; 2024: 60; 2025: 35; 2026: 22.

301 papers.

Sort:

By area:

  1. 2020Is Pessimism Provably Efficient for Offline RL?cited by 362
  2. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  3. 2021Urban Driver: Learning to Drive from Real-world Demonstrations Using Policy Gradientscited by 122
  4. 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
  5. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 103
  6. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  7. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  8. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 80
  9. 2021Federated Reinforcement Learning: Techniques, Applications, and Open Challengescited by 79
  10. 2021Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificatescited by 71
  11. 2021SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policiescited by 68
  12. 2021On the Convergence and Sample Efficiency of Variance-Reduced Policy Gradient Methodcited by 67
38 more
  1. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  2. 2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 66
  3. 2021Safe Multi-Agent Reinforcement Learning via Shieldingcited by 60
  4. 2021A Survey of Deep RL and IL for Autonomous Driving Policy Learningcited by 59
  5. 2022Independent Policy Gradient for Large-Scale Markov Potential Games: Sharper Rates, Function Approximation, and Game-Agnostic Convergencecited by 54
  6. 2021Decentralized Q-Learning in Zero-sum Markov Gamescited by 53
  7. 2025MARFT: Multi-Agent Reinforcement Fine-Tuningcited by 51
  8. 2021The Power of Exploiter: Provable Multi-Agent RL in Large State Spacescited by 48
  9. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 45
  10. 2022The Complexity of Markov Equilibrium in Stochastic Gamescited by 45
  11. 2021Towards General Function Approximation in Zero-Sum Markov Gamescited by 42
  12. 2021Provably Efficient Reinforcement Learning in Decentralized General-Sum Markov Gamescited by 42
  13. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 42
  14. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  15. 2024AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learningcited by 40
  16. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  17. 2025Multi-Agent Coordination across Diverse Applications: A Surveycited by 37
  18. 2021On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learningcited by 35
  19. 2021Deep reinforcement learning in medical imaging: A literature reviewcited by 34
  20. 2021Gradient play in stochastic games: stationary points, convergence, and sample complexitycited by 32
  21. 2021Multi-Agent Constrained Policy Optimisationcited by 32
  22. 2021Offline Pre-trained Multi-Agent Decision Transformer: One Big Sequence Model Tackles All SMAC Taskscited by 32
  23. 2019Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Drivingcited by 30
  24. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30
  25. 2021PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulationcited by 29
  26. 2022Provably Efficient Offline Multi-agent Reinforcement Learning via Strategy-wise Bonuscited by 29
  27. 2023A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergencecited by 29
  28. 2023Breaking the Curse of Multiagency: Provably Efficient Decentralized Multi-Agent RL with Function Approximationcited by 29
  29. 2022Safe Real-World Autonomous Driving by Learning to Predict and Plan with a Mixture of Expertscited by 28
  30. 2021Mean-Field Multi-Agent Reinforcement Learning: A Decentralized Network Approachcited by 27
  31. 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
  32. 2022Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processescited by 26
  33. 2022Sample-Efficient Reinforcement Learning of Partially Observable Markov Gamescited by 26
  34. 2021Almost Optimal Algorithms for Two-player Zero-Sum Linear Mixture Markov Gamescited by 25
  35. 2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attackscited by 25
  36. 2023Learning Realistic Traffic Agents in Closed-loopcited by 24
  37. 2025Discrete Diffusion for Reflective Vision-Language-Action Models in Autonomous Drivingcited by 24
  38. 2022Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processescited by 23

50 most cited of 301 All 301 (JSON)