mlsanityData to 7 Oct 2026

Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving

arXiv 2509.04712 ↗First posted 4 Sep 2025cs.RO cs.AI cs.LGcited by 0

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Automated vehicle control using reinforcement learning (RL) has attracted significant attention due to its potential to learn driving policies through environment interaction. However, RL agents often face training challenges in sample efficiency and effective exploration, making it difficult to discover an optimal driving strategy. To address these issues, we propose guiding the RL driving agent with a demonstration policy that need not be a highly optimized or expert-level controller. Specifically, we integrate a rule-based lane change controller with the Soft Actor Critic (SAC) algorithm to enhance exploration and learning efficiency. Our approach demonstrates improved driving performance and can be extended to other driving scenarios that can similarly benefit from demonstration-based guidance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 22 of its 41 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

22 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Prioritized Experience-based Reinforcement Learning with Human Guidance for Autonomous Driving26 Sep
  2. 2021Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving19 Mar

Other references · 20

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2016End to End Learning for Self-Driving Carscited by 1,000
  6. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  7. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  8. 2018Reward learning from human preferences and demonstrations in Ataricited by 314
  9. 2018Learning to Drive in a Daycited by 266
  10. 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
  11. 2019Urban Driving with Conditional Imitation Learningcited by 84
  12. 2017Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Carcited by 65
8 more
  1. 2019Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraintscited by 24
  2. 2020Learning hierarchical behavior and motion planning for autonomous drivingcited by 15
  3. 2019Driving Decision and Control for Autonomous Lane Change based on Deep Reinforcement Learningcited by 14
  4. 2019Driving Intention Recognition and Lane Change Prediction on the Highwaycited by 7
  5. 2020Integrating Deep Reinforcement Learning with Model-based Path Planners for Automated Drivingcited by 7
  6. 2019A Hierarchical Architecture for Sequential Decision-Making in Autonomous Driving using Deep Reinforcement Learningcited by 4
  7. 2020Combining Reinforcement Learning with Model Predictive Control for On-Ramp Mergingcited by 3
  8. 2025Extensive Exploration in Complex Traffic Scenarios using Hierarchical Reinforcement Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Improved Deep Reinforcement Learning with Expert Demonstrations for Urban Autonomous Driving
  2. 2025Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning
  3. 2025From Learning to Mastery: Achieving Safe and Efficient Real-World Autonomous Driving with Human-In-The-Loop Reinforcement Learning
  4. 2025A Comprehensive Review of Reinforcement Learning for Autonomous Driving in the CARLA Simulator
  5. 2020Deep Reinforcement Learning for Autonomous Driving: A Survey
  6. 2020Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous Driving
  7. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Survey
  8. 2025A Survey of Reinforcement Learning-Based Motion Planning for Autonomous Driving: Lessons Learned from a Driving Task Perspective