A Benchmark Environment for Offline Reinforcement Learning in Racing Games
On the map: Offline RL and imitation learning from data → Offline RL algorithms
Abstract (arXiv)
Offline Reinforcement Learning (ORL) is a promising approach to reduce the high sample complexity of traditional Reinforcement Learning (RL) by eliminating the need for continuous environmental interactions. ORL exploits a dataset of pre-collected transitions and thus expands the range of application of RL to tasks in which the excessive environment queries increase training time and decrease efficiency, such as in modern AAA games. This paper introduces OfflineMania a novel environment for ORL research. It is inspired by the iconic TrackMania series and developed using the Unity 3D game engine. The environment simulates a single-agent racing game in which the objective is to complete the track through optimal navigation. We provide a variety of datasets to assess ORL performance. These datasets, created from policies of varying ability and in different sizes, aim to offer a challenging testbed for algorithm development and evaluation. We further establish a set of baselines for a range of Online RL, ORL, and hybrid Offline to Online RL approaches using our environment.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 18 of its 23 references that are in the map. First posted on arXiv 12 Jul 2024.
Builds on
18 of 23 references.
Closest ancestors · 8
Same direction, newest first
- 2023Small Dataset, Big Gains: Enhancing Reinforcement Learning by Offline Pre-Training with Model Based Augmentation15 Dec
- 2023Policy Expansion for Bridging Offline-to-Online Reinforcement Learning2 Feb
- 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
- 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble1 Jul
- 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
- 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
- 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
Other references · 10
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
- 2018Unity: A General Platform for Intelligent Agentscited by 232
- 2022Jump-Start Reinforcement Learningcited by 102
- 2020Super-Human Performance in Gran Turismo Sport Using Deep Reinforcement Learningcited by 60
- 2021Augmenting Automated Game Testing with Deep Reinforcement Learningcited by 38
- 2022Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learningcited by 13
- 2023Learning Computational Efficient Bots with Costly Featurescited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2023Using Offline Data to Speed Up Reinforcement Learning in Procedurally Generated Environments
- 2022Bridging the Gap Between Offline and Online Reinforcement Learning Evaluation Methodologies
- 2020Offline Reinforcement Learning Hands-On
- 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning
- 2023Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions
- 2023The Generalization Gap in Offline Reinforcement Learning
- 2024Benchmarks for Reinforcement Learning with Biased Offline Data and Imperfect Simulators
- 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems