mlsanityData to 7 Oct 2026

Learning to Control Autonomous Fleets from Observation via Offline Reinforcement Learning

arXiv 2302.14833 ↗First posted 28 Feb 2023eess.SY cs.LG cs.ROcited by 2

On the map: Offline RL and imitation learning from data → Imitation and offline RL for driving

Abstract (arXiv)

Autonomous Mobility-on-Demand (AMoD) systems are an evolving mode of transportation in which a centrally coordinated fleet of self-driving vehicles dynamically serves travel requests. The control of these systems is typically formulated as a large network optimization problem, and reinforcement learning (RL) has recently emerged as a promising approach to solve the open challenges in this space. Recent centralized RL approaches focus on learning from online data, ignoring the per-sample-cost of interactions within real-world transportation systems. To address these limitations, we propose to formalize the control of AMoD systems through the lens of offline reinforcement learning and learn effective control strategies using solely offline data, which is readily available to current mobility operators. We further investigate design decisions and provide empirical evidence based on data from real-world mobility systems showing how offline learning allows to recover AMoD control policies that (i) exhibit performance on par with online methods, (ii) allow for sample-efficient online fine-tuning and (iii) eliminate the need for complex simulation environments. Crucially, this paper demonstrates that offline RL is a promising paradigm for the application of RL-based solutions within economically-critical systems, such as mobility systems.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data. Builds on 36 of its 69 references that are in the map. First posted on arXiv 28 Feb 2023.

Builds on

36 of 69 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Offline Reinforcement Learning for Road Traffic Control7 Jan
  2. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancement13 Oct

Other references · 34

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  7. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  8. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  9. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  10. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  11. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  12. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
22 more
  1. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  2. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  3. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  4. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
  5. 2019Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learningcited by 121
  6. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
  7. 2018Offline A/B testing for Recommender Systemscited by 94
  8. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
  9. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learningcited by 63
  10. 2021Reinforcement Learning for Ridesharing: An Extended Surveycited by 35
  11. 2019Deep Reinforcement Learning for Multi-Driver Vehicle Dispatching and Repositioning Problemcited by 29
  12. 2021Graph Neural Network Reinforcement Learning for Autonomous Mobility-on-Demand Systemscited by 27
  13. 2014Control of Robotic Mobility-On-Demand Systems: a Queueing-Theoretical Perspectivecited by 24
  14. 2017Data-Driven Model Predictive Control of Autonomous Mobility-on-Demand Systemscited by 23
  15. 2021Real-world Ride-hailing Vehicle Repositioning using Deep Reinforcement Learningcited by 22
  16. 2021Value Function is All You Need: A Unified Learning Framework for Ride Hailing Platformscited by 16
  17. 2022Transferring Multi-Agent Reinforcement Learning Policies for Autonomous Driving using Sim-to-Realcited by 16
  18. 2022Graph Meta-Reinforcement Learning for Transferable Autonomous Mobility-on-Demandcited by 14
  19. 2020Scalable Deep Reinforcement Learning for Ride-Hailingcited by 12
  20. 2022Reinforcement Learning in the Wild: Scalable RL Dispatching Algorithm Deployed in Ridehailing Marketplacecited by 11
  21. 2020Multi-Objective Vehicle Rebalancing for Ridehailing System using a Reinforcement Learning Approachcited by 4
  22. 2022Spatio-temporal Incentives Optimization for Ride-hailing Services with Offline Deep Reinforcement Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Atomic Proximal Policy Optimization for Electric Robo-Taxi Dispatch and Charger Allocationcited by 1
  2. 2025Offline Reinforcement Learning for Microgrid Voltage Regulationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021UMBRELLA: Uncertainty-Aware Model-Based Offline Reinforcement Learning Leveraging Planning
  2. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learning
  3. 2025A Tutorial: An Intuitive Explanation of Offline Reinforcement Learning Theory
  4. 2023Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstration
  5. 2023Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills
  6. 2024AD4RL: Autonomous Driving Benchmarks for Offline Reinforcement Learning with Value-based Dataset
  7. 2025From Imitation to Optimization: A Comparative Study of Offline Learning for Autonomous Driving
  8. 2024Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Study