mlsanityData to 7 Oct 2026

Goal-constrained Sparse Reinforcement Learning for End-to-End Driving

arXiv 2103.09189 ↗First posted 16 Mar 2021cs.RO cs.AI cs.CVcited by 3

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Deep reinforcement Learning for end-to-end driving is limited by the need of complex reward engineering. Sparse rewards can circumvent this challenge but suffers from long training time and leads to sub-optimal policy. In this work, we explore full-control driving with only goal-constrained sparse reward and propose a curriculum learning approach for end-to-end driving using only navigation view maps that benefit from small virtual-to-real domain gap. To address the complexity of multiple driving policies, we learn concurrent individual policies selected at inference by a navigation system. We demonstrate the ability of our proposal to generalize on unseen road layout, and to drive significantly longer than in the training.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 38 of its 61 references that are in the map. First posted on arXiv 16 Mar 2021.

Builds on

38 of 61 references.

References · 38

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  6. 2018World Modelscited by 1,444
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  8. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  9. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  10. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  11. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  12. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
26 more
  1. 2018Learning to Drive in a Daycited by 266
  2. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  3. 2017Automated Curriculum Learning for Neural Networkscited by 251
  4. 2017Teacher-Student Curriculum Learningcited by 197
  5. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 164
  6. 2017Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Playcited by 139
  7. 2020A Survey of End-to-End Driving: Architectures and Training Methodscited by 123
  8. 2018Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networkscited by 113
  9. 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 113
  10. 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 112
  11. 2020Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learningcited by 99
  12. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
  13. 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 79
  14. 2019Keeping Your Distance: Solving Sparse Reward Tasks Using Self-Balancing Shaped Rewardscited by 76
  15. 2019Dealing with Sparse Rewards in Reinforcement Learningcited by 40
  16. 2019Learning to Generalize from Sparse and Underspecified Rewardscited by 39
  17. 2018End-to-End Race Driving with Deep Reinforcement Learningcited by 38
  18. 2018Deep Reinforcement Learning for Autonomous Drivingcited by 36
  19. 2018Urban Driving with Multi-Objective Deep Reinforcement Learningcited by 30
  20. 2019Automated curricula through setter-solver interactionscited by 26
  21. 2019Autonomous Highway Driving using Deep Reinforcement Learningcited by 22
  22. 2018End to End Vehicle Lateral Control Using a Single Fisheye Cameracited by 13
  23. 2020Learning Accurate and Human-Like Driving using Semantic Maps and Attentioncited by 10
  24. 2020Robot Navigation with Map-Based Deep Reinforcement Learningcited by 9
  25. 2019Ctrl-Z: Recovering from Instability in Reinforcement Learningcited by 6
  26. 2019Learning sparse representations in reinforcement learningcited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 0; 2024: 1.

3 papers.

  1. 2022Handling Sparse Rewards in Reinforcement Learning Using Model Predictive Controlcited by 4
  2. 2022Automatic Evaluation of Excavator Operators using Learned Reward Functionscited by 0
  3. 2024Reinforcement Learning from Bagged Rewardcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Driving Beyond Privilege: Distilling Dense-Reward Knowledge into Sparse-Reward Policies
  2. 2026CORAL: Curriculum-Optimized Reward Adaptation for LiDAR-Based Goal-Directed Urban Driving
  3. 2022Learning to Drive Using Sparse Imitation Reinforcement Learning
  4. 2025CaRL: Learning Scalable Planning Policies with Simple Rewards
  5. 2022State Dropout-Based Curriculum Reinforcement Learning for Self-Driving at Unsignalized Intersections
  6. 2026Efficient Multi-Modal Planning with Reward-Guided Preference Optimization for Autonomous Driving
  7. 2019Driving Style Encoder: Situational Reward Adaptation for General-Purpose Planning in Automated Driving
  8. 2023Integration of Reinforcement Learning Based Behavior Planning With Sampling Based Motion Planning for Automated Driving