mlsanityData to 7 Oct 2026

Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors

arXiv 2305.04412 ↗First posted 8 May 2023cs.RO cs.AI cs.LGcited by 36

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

When autonomous vehicles are deployed on public roads, they will encounter countless and diverse driving situations. Many manually designed driving policies are difficult to scale to the real world. Fortunately, reinforcement learning has shown great success in many tasks by automatic trial and error. However, when it comes to autonomous driving in interactive dense traffic, RL agents either fail to learn reasonable performance or necessitate a large amount of data. Our insight is that when humans learn to drive, they will 1) make decisions over the high-level skill space instead of the low-level control space and 2) leverage expert prior knowledge rather than learning from scratch. Inspired by this, we propose ASAP-RL, an efficient reinforcement learning algorithm for autonomous driving that simultaneously leverages motion skills and expert priors. We first parameterized motion skills, which are diverse enough to cover various complex driving scenarios and situations. A skill parameter inverse recovery method is proposed to convert expert demonstrations from control space to skill space. A simple but effective double initialization technique is proposed to leverage expert priors while bypassing the issue of expert suboptimality and early performance degradation. We validate our proposed method on interactive dense-traffic driving tasks given simple and sparse rewards. Experimental results show that our method can lead to higher learning efficiency and better driving performance relative to previous methods that exploit skills and priors differently. Code is open-sourced to facilitate further research.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 34 of its 51 references that are in the map. First posted on arXiv 8 May 2023.

Builds on

34 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skills24 Sep
  2. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep
  3. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving18 Feb

Other references · 31

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  4. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  5. 2016The Option-Critic Architecturecited by 433
  6. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  7. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  8. 2021Bellman Eluder Dimension: New Rich Classes of RL Problems, and Sample-Efficient Algorithmscited by 222
  9. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  10. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
  11. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  12. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 99
19 more
  1. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 96
  2. 2018Neural probabilistic motor primitives for humanoid controlcited by 96
  3. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  4. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
  5. 2019Variable Impedance Control in End-Effector Space: An Action Space for Reinforcement Learning in Contact-Rich Taskscited by 56
  6. 2022Reinforcement Learning with Sparse Rewards using Guidance from Offline Demonstrationcited by 54
  7. 2020Learning compositional models of robot skills for task and motion planningcited by 47
  8. 2018How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Predictioncited by 46
  9. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  10. 2022Social Interactions for Autonomous Driving: A Review and Perspectivescited by 27
  11. 2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26
  12. 2021EPSILON: An Efficient Planning System for Automated Vehicles in Highly Interactive Environmentscited by 22
  13. 2019Keyframing the Future: Keyframe Discovery for Visual Prediction and Planningcited by 20
  14. 2022Transferable and Adaptable Driving Behavior Predictioncited by 20
  15. 2020Socially-Compatible Behavior Design of Autonomous Vehicles with Verification on Real Human Datacited by 18
  16. 2016Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networkscited by 15
  17. 2021Hierarchical Adaptable and Transferable Networks (HATN) for Driving Behavior Predictioncited by 10
  18. 2020Efficient Uncertainty-aware Decision-making for Automated Driving Using Guided Branchingcited by 8
  19. 2018CompILE: Compositional Imitation Learning and Executioncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 9; 2025: 16; 2026: 5.

36 papers.

Sort:

By area:

  1. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  2. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  3. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Drivingcited by 35
  4. 2024SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Predictioncited by 33
  5. 2025CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-scale Reinforcement Learning in Autonomous Drivingcited by 31
  6. 2025IRL-VLA: Training an Vision-Language-Action Policy via Reward World Modelcited by 31
  7. 2023Open-sourced Data Ecosystem in Autonomous Driving: the Present and Futurecited by 22
  8. 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Drivingcited by 20
  9. 2024Optimizing Autonomous Driving for Safety: A Human-Centric Approach with LLM-Enhanced RLHFcited by 18
  10. 2024A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Drivingcited by 17
  11. 2023A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learningcited by 15
  12. 2024Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Searchcited by 13
24 more
  1. 2024SmartPretrain: Model-Agnostic and Dataset-Agnostic Representation Learning for Motion Predictioncited by 7
  2. 2025Open-World Skill Discovery from Unsegmented Demonstrationscited by 7
  3. 2023PathRL: An End-to-End Path Generation Method for Collision Avoidance via Deep Reinforcement Learningcited by 5
  4. 2025Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectivescited by 5
  5. 2023Imagination-Augmented Hierarchical Reinforcement Learning for Safe and Interactive Autonomous Driving in Urban Environmentscited by 3
  6. 2025DriveMind: A Dual Visual Language Model-based Reinforcement Learning Framework for Autonomous Drivingcited by 3
  7. 2025ForeSight: Multi-View Streaming Joint Object Detection and Trajectory Forecastingcited by 3
  8. 2025Large-Scale Mixed-Traffic and Intersection Control using Multi-agent Reinforcement Learningcited by 2
  9. 2023Integrating Expert Guidance for Efficient Learning of Safe Overtaking in Autonomous Driving Using Deep Reinforcement Learningcited by 1
  10. 2024Act Better by Timing: A timing-Aware Reinforcement Learning for Autonomous Drivingcited by 1
  11. 2024Safe CoR: A Dual-Expert Approach to Integrating Imitation Learning and Safe Reinforcement Learning Using Constraint Rewardscited by 1
  12. 2025Scenario-Based Hierarchical Reinforcement Learning for Automated Driving Decision Makingcited by 1
  13. 2026CosmosAlign: Adapting a World Foundation Model for Generative Traffic Video Forecastingcited by 1
  14. 2025AI Agents and Hard Choicescited by 0
  15. 2025Interactive Double Deep Q-network: Integrating Human Interventions and Evaluative Predictions in Reinforcement Learning of Autonomous Drivingcited by 0
  16. 2025Learning-based Autonomous Oversteer Control and Collision Avoidancecited by 0
  17. 2025Multi-Timescale Hierarchical Reinforcement Learning for Unified Behavior and Control of Autonomous Drivingcited by 0
  18. 2025ObjectRL: An Object-Oriented Reinforcement Learning Codebasecited by 0
  19. 2025Decision Transformer-Based Drone Trajectory Planning with Dynamic Safety-Efficiency Trade-Offscited by 0
  20. 2025Harnessing Bounded-Support Evolution Strategies for Policy Refinementcited by 0
  21. 2025Vehicle Dynamics Embedded World Models for Autonomous Drivingcited by 0
  22. 2026Intrinsic Vicarious Conditioning for Deep Reinforcement Learningcited by 0
  23. 2026MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planningcited by 0
  24. 2026Meta-Multi-Agent Reinforcement Learning for Fast Adaptation of Interactive Policies with Applications to Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning
  2. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective
  3. 2023Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approach
  4. 2024Autonomous Algorithm for Training Autonomous Vehicles with Minimal Human Intervention
  5. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
  6. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing
  7. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving
  8. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey