mlsanityData to 7 Oct 2026

Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flow

arXiv 2304.12821 ↗First posted 25 Apr 2023cs.ROcited by 0

On the map: End-to-end autonomous driving → Imitation-learning end-to-end planners

Abstract (arXiv)

Acquiring driving policies that can transfer to unseen environments is challenging when driving in dense traffic flows. The design of traffic flow is essential and previous studies are unable to balance interaction and safety-criticism. To tackle this problem, we propose a socially adversarial traffic flow. We propose a Contextual Partially-Observable Stochastic Game to model traffic flow and assign Social Value Orientation (SVO) as context. We then adopt a two-stage framework. In Stage 1, each agent in our socially-aware traffic flow is driven by a hierarchical policy where upper-level policy communicates genuine SVOs of all agents, which the lower-level policy takes as input. In Stage 2, each agent in the socially adversarial traffic flow is driven by the hierarchical policy where upper-level communicates mistaken SVOs, taken by the lower-level policy trained in Stage 1. Driving policy is adversarially trained through a zero-sum game formulation with upper-level policies, resulting in a policy with enhanced zero-shot transfer capability to unseen traffic flows. Comprehensive experiments on cross-validation verify the superior zero-shot transfer performance of our method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation-learning end-to-end planners, one of 9 directions in End-to-end autonomous driving. Builds on 58 of its 81 references that are in the map. First posted on arXiv 25 Apr 2023.

Builds on

58 of 81 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022KING: Generating Safety-Critical Driving Scenarios for Robust Imitation via Kinematics Gradients28 Apr
  2. 2021Closing the Planning-Learning Loop with Application to Autonomous Driving11 Jan

Other references · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  8. 2017Deep Setscited by 1,394
  9. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  10. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 528
  11. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  12. 2017Robust Adversarial Reinforcement Learningcited by 439
38 more
  1. 2020Learning Lane Graph Representations for Motion Forecastingcited by 380
  2. 2019Learning by Cheatingcited by 361
  3. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  4. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  5. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  6. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 281
  7. 2015Contextual Markov Decision Processescited by 213
  8. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  9. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  10. 2019End-to-End Model-Free Reinforcement Learning for Urban Driving using Implicit Affordancescited by 147
  11. 2016EPOpt: Learning Robust Neural Network Policies Using Model Ensemblescited by 136
  12. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  13. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
  14. 2018Deep Imitative Models for Flexible Inference, Planning, and Controlcited by 75
  15. 2020Robust Deep Reinforcement Learning through Adversarial Losscited by 73
  16. 2017Flow: A Modular Learning Framework for Mixed Autonomy Trafficcited by 67
  17. 2020Robust Reinforcement Learning using Adversarial Populationscited by 64
  18. 2019Risk Averse Robust Adversarial Reinforcement Learningcited by 54
  19. 2019Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learningcited by 53
  20. 2020Adversarial Evaluation of Autonomous Vehicles in Lane-Change Scenarioscited by 42
  21. 2017Dissipation of stop-and-go waves via control of autonomous vehicles: Field experimentscited by 41
  22. 2019SUMMIT: A Simulator for Urban Driving in Massive Mixed Trafficcited by 39
  23. 2019Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Drivingcited by 36
  24. 2019Driving in Dense Traffic with Model-Free Reinforcement Learningcited by 33
  25. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  26. 2021Social Coordination and Altruism in Autonomous Drivingcited by 28
  27. 2022Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Trainingcited by 26
  28. 2019Cooperation-Aware Reinforcement Learning for Merging in Dense Trafficcited by 20
  29. 2022Interaction-aware Decision-making for Automated Vehicles using Social Value Orientationcited by 20
  30. 2019Investigating Generalisation in Continuous Deep Reinforcement Learningcited by 18
  31. 2021Cooperative Autonomous Vehicles that Sympathize with Human Driverscited by 17
  32. 2020Learning hierarchical behavior and motion planning for autonomous drivingcited by 15
  33. 2019LeTS-Drive: Driving in a Crowd by Learning from Tree Searchcited by 13
  34. 2020Improving Generalization of Reinforcement Learning with Minimax Distributional Soft Actor-Criticcited by 12
  35. 2021Scalable Multiagent Driving Policies For Reducing Traffic Congestioncited by 11
  36. 2021Altruistic Maneuver Planning for Cooperative Autonomous Vehicles Using Multi-agent Advantage Actor-Criticcited by 11
  37. 2020Reinforcement Learning with Iterative Reasoning for Merging in Dense Trafficcited by 10
  38. 2020Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectorycited by 8

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Robust Driving Policy Learning with Guided Meta Reinforcement Learning
  2. 2024Learning to Drive via Asymmetric Self-Play
  3. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environments
  4. 2025DecompGAIL: Learning Realistic Traffic Behaviors with Decomposed Multi-Agent Generative Adversarial Imitation Learning
  5. 2025Toward Efficient and Robust Behavior Models for Multi-Agent Driving Simulation
  6. 2026Aligning Microscopic Vehicle and Macroscopic Traffic Statistics: Reconstructing Driving Behavior from Partial Data
  7. 2022Improve Generalization of Driving Policy at Signalized Intersections with Adversarial Learning
  8. 2025Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach