# MOVI: A Model-Free Approach to Dynamic Fleet Management

arXiv: https://arxiv.org/abs/1804.04758 · HTML: https://mlsanity.com/paper/1804.04758/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2018.

## Built on by (most cited first, 23 of 23)

- [Reinforcement Learning for Ridesharing: An Extended Survey](https://mlsanity.com/paper/2105.01099/) (2021)
- [Real-world Ride-hailing Vehicle Repositioning using Deep Reinforcement Learning](https://mlsanity.com/paper/2103.04555/) (2021)
- [Value Function is All You Need: A Unified Learning Framework for Ride Hailing Platforms](https://mlsanity.com/paper/2105.08791/) (2021)
- [AdaPool: A Diurnal-Adaptive Fleet Management Framework using Model-Free Deep Reinforcement Learning and Change Point Detection](https://mlsanity.com/paper/2104.00203/) (2021)
- [i-Rebalance: Personalized Vehicle Repositioning for Supply Demand Balance](https://mlsanity.com/paper/2401.04429/) (2024)
- [Equilibrium Inverse Reinforcement Learning for Ride-hailing Vehicle Network](https://mlsanity.com/paper/2102.06854/) (2021)
- [Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey](https://mlsanity.com/paper/2108.04462/) (2021)
- [A Survey of Machine Learning-Based Ride-Hailing Planning](https://mlsanity.com/paper/2303.14646/) (2023)
- [Mutual Information as Intrinsic Reward of Reinforcement Learning Agents for On-demand Ride Pooling](https://mlsanity.com/paper/2312.15195/) (2023)
- [Coordinating Ride-Pooling with Public Transit using Reward-Guided Conservative Q-Learning: An Offline Training and Online Fine-Tuning Reinforcement Learning Framework](https://mlsanity.com/paper/2501.14199/) (2025)
- [A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications](https://mlsanity.com/paper/2210.14524/) (2022)
- [DeepFreight: Integrating Deep Reinforcement Learning and Mixed Integer Programming for Multi-transfer Truck Freight Delivery](https://mlsanity.com/paper/2103.03450/) (2021)
- [Off-line approximate dynamic programming for the vehicle routing problem with a highly variable customer basis and stochastic demands](https://mlsanity.com/paper/2109.10200/) (2021)
- [Online Location Planning for AI-Defined Vehicles: Optimizing Joint Tasks of Order Serving and Spatio-Temporal Heterogeneous Model Fine-Tuning](https://mlsanity.com/paper/2502.04399/) (2025)
- [CoordiQ : Coordinated Q-learning for Electric Vehicle Charging Recommendation](https://mlsanity.com/paper/2102.00847/) (2021)
- [H-TD2: Hybrid Temporal Difference Learning for Adaptive Urban Taxi Dispatch](https://mlsanity.com/paper/2105.02138/) (2021)
- [Learning Model-Based Vehicle-Relocation Decisions for Real-Time Ride-Sharing: Hybridizing Learning and Optimization](https://mlsanity.com/paper/2105.13461/) (2021)
- [Learning Model Predictive Controllers for Real-Time Ride-Hailing Vehicle Relocation and Pricing Decisions](https://mlsanity.com/paper/2111.03204/) (2021)
- [Processing Network Controls via Deep Reinforcement Learning](https://mlsanity.com/paper/2205.02119/) (2022)
- [Towards More Efficient Shared Autonomous Mobility: A Learning-Based Fleet Repositioning Approach](https://mlsanity.com/paper/2210.08659/) (2022)
- [Deep-Dispatch: A Deep Reinforcement Learning-Based Vehicle Dispatch Algorithm for Advanced Air Mobility](https://mlsanity.com/paper/2312.10809/) (2023)
- [Scalable Ride-Sourcing Vehicle Rebalancing with Service Accessibility Guarantee: A Constrained Mean-Field Reinforcement Learning Approach](https://mlsanity.com/paper/2503.24183/) (2025)
- [A Deep Reinforcement Learning Algorithm for the Vehicle Routing Problem with Stochastic Demands and Outsourcing](https://mlsanity.com/paper/2607.16875/) (2026)

## Abstract (arXiv)

Modern vehicle fleets, e.g., for ridesharing platforms and taxi companies, can reduce passengers' waiting times by proactively dispatching vehicles to locations where pickup requests are anticipated in the future. Yet it is unclear how to best do this: optimal dispatching requires optimizing over several sources of uncertainty, including vehicles' travel times to their dispatched locations, as well as coordinating between vehicles so that they do not attempt to pick up the same passenger. While prior works have developed models for this uncertainty and used them to optimize dispatch policies, in this work we introduce a model-free approach. Specifically, we propose MOVI, a Deep Q-network (DQN)-based framework that directly learns the optimal vehicle dispatch policy. Since DQNs scale poorly with a large number of possible dispatches, we streamline our DQN training and suppose that each individual vehicle independently learns its own optimal policy, ensuring scalability at the cost of less coordination between vehicles. We then formulate a centralized receding-horizon control (RHC) policy to compare with our DQN policies. To compare these policies, we design and build MOVI as a large-scale realistic simulator based on 15 million taxi trip records that simulates policy-agnostic responses to dispatch decisions. We show that the DQN dispatch policy reduces the number of unserviced requests by 76% compared to without dispatch and 20% compared to the RHC approach, emphasizing the benefits of a model-free approach and suggesting that there is limited value to coordinating vehicle actions. This finding may help to explain the success of ridesharing platforms, for which drivers make individual decisions.