mlsanityData to 7 Oct 2026

AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset

arXiv 2404.02429 ↗First posted 3 Apr 2024cs.LG cs.AIcited by 14

On the map: End-to-end autonomous driving → Reinforcement learning for driving

also in 1 more
Abstract (arXiv)

Offline reinforcement learning has emerged as a promising technology by enhancing its practicality through the use of pre-collected large datasets. Despite its practical benefits, most algorithm development research in offline reinforcement learning still relies on game tasks with synthetic datasets. To address such limitations, this paper provides autonomous driving datasets and benchmarks for offline reinforcement learning research. We provide 19 datasets, including real-world human driver's datasets, and seven popular offline reinforcement learning algorithms in three realistic driving scenarios. We also provide a unified decision-making process model that can operate effectively across different scenarios, serving as a reference framework in algorithm design. Our research lays the groundwork for further collaborations in the community to explore practical aspects of existing reinforcement learning methods. Dataset and codes can be found in https://sites.google.com/view/ad4rl.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 25 of its 41 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

25 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement13 Oct
  2. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep

Other references · 23

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  8. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  9. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  10. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  11. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  12. 2018An Algorithmic Perspective on Imitation Learningcited by 455
11 more
  1. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 297
  2. 2018Deep learning in agriculture: A surveycited by 160
  3. 2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
  4. 2018CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-drivingcited by 96
  5. 2020Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Drivingcited by 73
  6. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
  7. 2017Flow: A Modular Learning Framework for Mixed Autonomy Trafficcited by 67
  8. 2022When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learningcited by 59
  9. 2022Safe Real-World Autonomous Driving by Learning to Predict and Plan with a Mixture of Expertscited by 28
  10. 2022Control-Aware Prediction Objectives for Autonomous Drivingcited by 18
  11. 2018Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehiclescited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 6.

14 papers.

Sort:

By area:

  1. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  2. 2024Solving Continual Offline RL through Selective Weights Activation on Aligned Spacescited by 6
  3. 2025Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learningcited by 5
  4. 2024Episodic Future Thinking Mechanism for Multi-agent Reinforcement Learningcited by 4
  5. 2025Human-Level Competitive Pok'emon via Scalable Offline Reinforcement Learning with Transformerscited by 4
  6. 2025Offline Reinforcement Learning using Human-Aligned Reward Labeling for Autonomous Emergency Braking in Occluded Pedestrian Crossingcited by 1
  7. 2026Multi2: Hierarchical Multi-Agent Decision-Making with LLM-Based Agents in Interactive Environmentscited by 1
  8. 2025PyTupli: A Scalable Infrastructure for Collaborative Offline Reinforcement Learning Projectscited by 0
  9. 2025LLM-Driven Policy Diffusion: Enhancing Generalization in Offline Reinforcement Learningcited by 0
  10. 2026Offline Reinforcement Learning of High-Quality Behaviors Under Robust Style Alignmentcited by 0
  11. 2026A Recipe for Stable Offline Multi-agent Reinforcement Learningcited by 0
  12. 2026From Virtual Environments to Real-World Trials: Emerging Trends in Autonomous Drivingcited by 0
2 more
  1. 2026Simple Actors and Deep Critics for Scalable Reinforcement Learningcited by 0
  2. 2026A Survey on End-to-End Autonomous Driving Training from the Perspectives of Data, Strategy, and Platformcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024D5RL: Diverse Datasets for Data-Driven Deep Reinforcement Learning
  2. 2023Improving and Benchmarking Offline Reinforcement Learning Algorithms
  3. 2024Benchmarks for Reinforcement Learning with Biased Offline Data and Imperfect Simulators
  4. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learning
  5. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learning
  6. 2025NeoRL-2: Near Real-World Benchmarks for Offline Reinforcement Learning with Extended Realistic Scenarios
  7. 2020RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning
  8. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learning