mlsanityData to 7 Oct 2026

Driving in Dense Traffic with Model-Free Reinforcement Learning

arXiv 1909.06710 ↗First posted 15 Sep 2019cs.RO cs.AI cs.LGcited by 33

On the map: World models → World models for autonomous driving

Abstract (arXiv)

Traditional planning and control methods could fail to find a feasible trajectory for an autonomous vehicle to execute amongst dense traffic on roads. This is because the obstacle-free volume in spacetime is very small in these scenarios for the vehicle to drive through. However, that does not mean the task is infeasible since human drivers are known to be able to drive amongst dense traffic by leveraging the cooperativeness of other drivers to open a gap. The traditional methods fail to take into account the fact that the actions taken by an agent affect the behaviour of other vehicles on the road. In this work, we rely on the ability of deep reinforcement learning to implicitly model such interactions and learn a continuous control policy over the action space of an autonomous vehicle. The application we consider requires our agent to negotiate and open a gap in the road in order to successfully merge or change lanes. Our policy learns to repeatedly probe into the target road lane while trying to find a safe spot to move in to. We compare against two model-predictive control-based algorithms and show that our policy outperforms them in simulation.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 2 of its 37 references that are in the map. First posted on arXiv 15 Sep 2019.

Builds on

2 of 37 references.

Closest ancestors · 1

Same direction, newest first

  1. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Driving20 Apr

Other references · 1

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 3; 2023: 8; 2024: 5; 2025: 7; 2026: 2.

33 papers.

Sort:

By area:

  1. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  2. 2023Motion Planning for Autonomous Driving: The State of the Art and Future Perspectivescited by 112
  3. 2021DRIVE: Deep Reinforced Accident Anticipation with Visual Explanationcited by 46
  4. 2021PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulationcited by 29
  5. 2022Challenges and Opportunities in Deep Reinforcement Learning with Graph Neural Networks: A Comprehensive review of Algorithms and Applicationscited by 27
  6. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  7. 2021Deep Structured Reactive Planningcited by 18
  8. 2021End-to-End Interactive Prediction and Planning with Optical Flow Distillation for Autonomous Drivingcited by 17
  9. 2021DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networkscited by 13
  10. 2022Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skillscited by 12
  11. 2023Towards Socially Responsive Autonomous Vehicles: A Reinforcement Learning Framework with Driving Priors and Coordination Awarenesscited by 12
  12. 2021Carl-Lead: Lidar-based End-to-End Autonomous Driving with Contrastive Deep Reinforcement Learningcited by 11
21 more
  1. 2024DDM-Lag : A Diffusion-based Decision-making Model for Autonomous Vehicles with Lagrangian Safety Enhancementcited by 11
  2. 2022Spatiotemporal Costmap Inference for MPC via Deep Inverse Reinforcement Learningcited by 10
  3. 2023Interactive Autonomous Navigation with Internal State Inference and Interactivity Estimationcited by 10
  4. 2021Learning Interaction-aware Guidance Policies for Motion Planning in Dense Traffic Scenarioscited by 7
  5. 2025Predictive Planner for Autonomous Driving with Consistency Modelscited by 7
  6. 2025RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policiescited by 6
  7. 2021Vision Transformer for Learning Driving Policies in Complex Multi-Agent Environmentscited by 5
  8. 2023Interaction-Aware Trajectory Planning for Autonomous Vehicles with Analytic Integration of Neural Networks into Model Predictive Controlcited by 5
  9. 2024Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Drivingcited by 4
  10. 2025Enhancing PPO with Trajectory-Aware Hybrid Policiescited by 4
  11. 2025HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Drivingcited by 4
  12. 2026Scaling Self-Play for End-to-End Drivingcited by 4
  13. 2026ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Drivingcited by 3
  14. 2023A Hierarchical Hybrid Learning Framework for Multi-agent Trajectory Predictioncited by 2
  15. 2023A Two-stage Based Social Preference Recognition in Multi-Agent Autonomous Driving Systemcited by 1
  16. 2025A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approachcited by 1
  17. 2023Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flowcited by 0
  18. 2024Safe Hybrid-Action Reinforcement Learning-Based Decision and Control for Discretionary Lane Changecited by 0
  19. 2024Multi-Task Lane-Free Driving Strategy for Connected and Automated Vehicles: A Multi-Agent Deep Reinforcement Learning Approachcited by 0
  20. 2025A Hybrid Input based Deep Reinforcement Learning for Lane Change Decision-Making of Autonomous Vehiclecited by 0
  21. 2025Learning to Drive Safely with Hybrid Optionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions
  2. 2021Affordance-based Reinforcement Learning for Urban Driving
  3. 2024Agent-Agnostic Centralized Training for Decentralized Multi-Agent Cooperative Driving
  4. 2021Learning to Robustly Negotiate Bi-Directional Lane Usage in High-Conflict Driving Scenarios
  5. 2024Traffic Smoothing Controllers for Autonomous Vehicles Using Deep Reinforcement Learning and Real-World Trajectory Data
  6. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
  7. 2020Meta Reinforcement Learning-Based Lane Change Strategy for Autonomous Vehicles
  8. 2023Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approach