mlsanityData to 13 Oct 2026

Learning robust driving policies without online exploration

arXiv 2103.08070 ↗First posted 15 Mar 2021cs.ROcited by 2

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

We propose a multi-time-scale predictive representation learning method to efficiently learn robust driving policies in an offline manner that generalize well to novel road geometries, and damaged and distracting lane conditions which are not covered in the offline training data. We show that our proposed representation learning method can be applied easily in an offline (batch) reinforcement learning setting demonstrating the ability to generalize well and efficiently under novel conditions compared to standard batch RL methods. Our proposed method utilizes training data collected entirely offline in the real-world which removes the need of intensive online explorations that impede applying deep reinforcement learning on real-world robot training. Various experiments were conducted in both simulator and real-world scenarios for the purpose of evaluation and analysis of our proposed claims.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 28 of its 63 references that are in the map. First posted on arXiv 15 Mar 2021.

Builds on

28 of 63 references.

References · 28

  1. 2015Continuous control with deep reinforcement learningcited by 3,290
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,221
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,889
  4. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,231
  5. 2015Prioritized Experience Replaycited by 1,078
  6. 2016End to End Learning for Self-Driving Carscited by 1,002
  7. 2017Deep Reinforcement Learning that Matterscited by 734
  8. 2019Solving Rubik's Cube with a Robot Handcited by 702
  9. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 699
  10. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 403
  11. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 393
  12. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
16 more
  1. 2016Successor Features for Transfer in Reinforcement Learningcited by 332
  2. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  3. 2019Challenges of Real-World Reinforcement Learningcited by 285
  4. 2016A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehiclescited by 253
  5. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 231
  6. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224
  7. 2018Generalization and Regularization in DQNcited by 117
  8. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 108
  9. 2017Hybrid Reward Architecture for Reinforcement Learningcited by 57
  10. 2019Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safetycited by 48
  11. 2019Robust Lane Detection from Continuous Driving Scenes Using Deep Neural Networkscited by 41
  12. 2019Importance Resampling for Off-policy Predictioncited by 26
  13. 2019Investigating Generalisation in Continuous Deep Reinforcement Learningcited by 18
  14. 2017Deep Steering: Learning End-to-End Driving Model from Spatial and Temporal Visual Cuescited by 9
  15. 2018Online Off-policy Predictioncited by 9
  16. 2020Perception as prediction using general value functions in autonomous driving applicationscited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agentscited by 7
  2. 2025PrivORL: Differentially Private Synthetic Dataset for Offline Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills
  2. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement
  3. 2022Optimizing Trajectories for Highway Driving with Offline Reinforcement Learning
  4. 2025From Imitation to Optimization: A Comparative Study of Offline Learning for Autonomous Driving
  5. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning
  6. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Driving
  7. 2023Pay Attention to How You Drive: Safe and Adaptive Model-Based Reinforcement Learning for Off-Road Driving
  8. 2025Pseudo-Expert Regularized Offline RL for End-to-End Autonomous Driving in Photorealistic Closed-Loop Environments