mlsanityData to 14 Oct 2026

Continual-RL for Generalization in Autonomous Racing on the RoboRacer Platform

arXiv 2607.24320 ↗First posted 27 Jul 2026cs.RO cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

A key challenge in modern robotics is to adapt to changing environments, a challenge that is exacerbated when simulations cannot encompass every possible real-world configuration, and therefore Reinforcement Learning (RL) in the physical world becomes necessary. Continual Reinforcement Learning provides the tools to address this challenge; however, both the frameworks and the methods remain underexplored. Autonomous Racing and in particular the RoboRacer competition provide a testing ground for such methods, as learning to drive on a new track-floor combination with the least amount of new experience naturally frames a continual learning problem. This work tries to address this gap by proposing a continual RL framework based on Continual Backpropagation that is able, with only real-world data, to train a generalistic policy on a set of tracks and then fine- tune it within 15 minutes to outperform classical controllers. Furthermore, a comparison method based on offline RL is proposed, and a simulation analysis of the plasticity properties of the methods is conducted.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 39 of its 51 references that are in the map. First posted on arXiv 27 Jul 2026.

Builds on

39 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025A Survey of Continual Reinforcement Learning27 Jun
  2. 2024SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning13 Oct

Other references · 37

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  3. 2018Experience Replay for Continual Learningcited by 1,095
  4. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  5. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  6. 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 215
  7. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 214
  8. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  9. 2023The Dormant Neuron Phenomenon in Deep Reinforcement Learningcited by 174
  10. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  11. 2023A Definition of Continual Reinforcement Learningcited by 101
  12. 2019CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicitycited by 95
25 more
  1. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 84
  2. 2023Maintaining Plasticity in Continual Learning via Regenerative Regularizationcited by 65
  3. 2020Smooth Exploration for Robotic Reinforcement Learningcited by 54
  4. 2017Optimization-Based Autonomous Racing of 1:43 Scale RC Carscited by 50
  5. 2021Autonomous Overtaking in Gran Turismo Sport Using Curriculum Reinforcement Learningcited by 45
  6. 2019ELF OpenGo: An Analysis and Open Reimplementation of AlphaZerocited by 39
  7. 2021CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agentscited by 38
  8. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Modelscited by 38
  9. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 37
  10. 2021Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racingcited by 26
  11. 2025Parkour in the Wild: Learning a General and Extensible Agile Locomotion Policy Using Multi-expert Distillation and RL Fine-tuningcited by 26
  12. 2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 22
  13. 2019Boosting Soft Actor-Critic: Emphasizing Recent Experience without Forgetting the Pastcited by 18
  14. 2023FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicingcited by 17
  15. 2020Circus ANYmal: A Quadruped Learning Dexterous Manipulation with Its Limbscited by 16
  16. 2022TUM autonomous motorsport: An autonomous racing software for the Indy Autonomous Challengecited by 14
  17. 2022Model- and Acceleration-based Pursuit Controller for High-Performance Autonomous Racingcited by 13
  18. 2024Unifying F1TENTH Autonomous Racing: Survey, Methods and Benchmarkscited by 8
  19. 2024ForzaETH Race Stack -- Scaled Autonomous Head-to-Head Racing on Fully Commercial off-the-Shelf Hardwarecited by 8
  20. 2024Bridging the Sim-to-Real Gap with Bayesian Inferencecited by 8
  21. 2024TinyLidarNet: 2D LiDAR-based End-to-End Deep Learning Model for F1TENTH Autonomous Racingcited by 6
  22. 2024Learning-Based On-Track System Identification for Scaled Autonomous Racing in Under a Minutecited by 5
  23. 2024Reference-Free Formula Drift with Reinforcement Learning: From Driving Data to Tire Energy-Inspired, Real-World Policiescited by 4
  24. 2023End-to-end Lidar-Driven Reinforcement Learning for Autonomous Racingcited by 3
  25. 2025Drive Fast, Learn Faster: On-Board RL for High Performance Autonomous Racingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Continual Reinforcement Learning for Cyber-Physical Systems: Lessons Learned and Open Challenges
  2. 2025Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review
  3. 2025On learning racing policies with reinforcement learning
  4. 2025RLPP: A Residual Method for Zero-Shot Real-World Autonomous Racing on Scaled Platforms
  5. 2025Demonstration-Guided Continual Reinforcement Learning in Dynamic Environments
  6. 2025Context-Aware Model-Based Reinforcement Learning for Autonomous Racing
  7. 2025The Cell Must Go On: Agar.io for Continual Reinforcement Learning
  8. 2026Adaptive Control in Autonomous Driving via Real-Time Recurrent RL