mlsanityData to 7 Oct 2026

F1tenth Autonomous Racing With Offline Reinforcement Learning Methods

arXiv 2408.04198 ↗First posted 8 Aug 2024cs.RO cs.SY eess.SYcited by 0

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

Autonomous racing serves as a critical platform for evaluating automated driving systems and enhancing vehicle mobility intelligence. This work investigates offline reinforcement learning methods to train agents within the dynamic F1tenth racing environment. The study begins by exploring the challenges of online training in the Austria race track environment, where agents consistently fail to complete the laps. Consequently, this research pivots towards an offline strategy, leveraging `expert' demonstration dataset to facilitate agent training. A waypoint-based suboptimal controller is developed to gather data with successful lap episodes. This data is then employed to train offline learning-based algorithms, with a subsequent analysis of the agents' cross-track performance, evaluating their zero-shot transferability from seen to unseen scenarios and their capacity to adapt to changes in environment dynamics. Beyond mere algorithm benchmarking in autonomous racing scenarios, this study also introduces and describes the machinery of our return-conditioned decision tree-based policy, comparing its performance with methods that employ fully connected neural networks, Transformers, and Diffusion Policies and highlighting some insights into method selection for training autonomous agents in driving interactions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 28 of its 41 references that are in the map. First posted on arXiv 8 Aug 2024.

Builds on

28 of 41 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  7. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  8. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  9. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  10. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  11. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  12. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
16 more
  1. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  2. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  3. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  4. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  5. 2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
  6. 2022CORL: Research-oriented Deep Offline Reinforcement Learning Librarycited by 117
  7. 2018Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Drivingcited by 90
  8. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  9. 2022Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racingcited by 56
  10. 2017Optimization-Based Autonomous Racing of 1:43 Scale RC Carscited by 47
  11. 2015An Empirical Evaluation of Deep Learning on Highway Drivingcited by 35
  12. 2021Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racingcited by 22
  13. 2019A Review of Tracking, Prediction and Decision Making Methods for Autonomous Drivingcited by 17
  14. 2020Long-term Prediction of Vehicle Behavior using Short-term Uncertainty-aware Trajectories and High-definition Mapscited by 5
  15. 2023High-speed Autonomous Racing using Trajectory-aided Deep Reinforcement Learningcited by 5
  16. 2024Solving Offline Reinforcement Learning with Decision Tree Regressioncited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Track based Offline Policy Learning for Overtaking Maneuvers with Autonomous Racecars
  2. 2024A Benchmark Environment for Offline Reinforcement Learning in Racing Games
  3. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
  4. 2021Autonomous Racing using a Hybrid Imitation-Reinforcement Learning Architecture
  5. 2024AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset
  6. 2023Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills
  7. 2022A Benchmark Comparison of Imitation Learning-based Control Policies for Autonomous Racing
  8. 2024A Simulation Benchmark for Autonomous Racing with Large-Scale Human Data