# A Deep Value-network Based Approach for Multi-Driver Order Dispatching

arXiv: https://arxiv.org/abs/2106.04493 · HTML: https://mlsanity.com/paper/2106.04493/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 28 references that are in the map.
- First posted on arXiv 8 Jun 2021.

## Builds on (most cited first, 7 of 7)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Parseval Networks: Improving Robustness to Adversarial Examples](https://mlsanity.com/paper/1704.08847/) (2017)
- [Control of Robotic Mobility-On-Demand Systems: a Queueing-Theoretical Perspective](https://mlsanity.com/paper/1404.4391/) (2014)
- [Lipschitz regularized Deep Neural Networks generalize and are adversarially robust](https://mlsanity.com/paper/1808.09540/) (2018)

## Built on by (most cited first, 30 of 45)

- [Showing Your Offline Reinforcement Learning Work: Online Evaluation Budget Matters](https://mlsanity.com/paper/2110.04156/) (2021)
- [Hybrid Multi-agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems](https://mlsanity.com/paper/2212.07313/) (2022)
- [Learning-based Online Optimization for Autonomous Mobility-on-Demand Fleet Control](https://mlsanity.com/paper/2302.03963/) (2023)
- [Reinforcement Learning in the Wild: Scalable RL Dispatching Algorithm Deployed in Ridehailing Marketplace](https://mlsanity.com/paper/2202.05118/) (2022)
- [A Better Match for Drivers and Riders: Reinforcement Learning at Lyft](https://mlsanity.com/paper/2310.13810/) (2023)
- [Policy Evaluation for Temporal and/or Spatial Dependent Experiments](https://mlsanity.com/paper/2202.10887/) (2022)
- [Global Rewards in Multi-Agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems](https://mlsanity.com/paper/2312.08884/) (2023)
- [An End-to-End Reinforcement Learning Based Approach for Micro-View Order-Dispatching in Ride-Hailing](https://mlsanity.com/paper/2408.10479/) (2024)
- [Risk-Sensitive Soft Actor-Critic for Robust Deep Reinforcement Learning under Distribution Shifts](https://mlsanity.com/paper/2402.09992/) (2024)
- [Unraveling the Interplay between Carryover Effects and Reward Autocorrelations in Switchback Experiments](https://mlsanity.com/paper/2403.17285/) (2024)
- [Combining Experimental and Historical Data for Policy Evaluation](https://mlsanity.com/paper/2406.00317/) (2024)
- [Neural-to-Tree Policy Distillation with Policy Improvement Criterion](https://mlsanity.com/paper/2108.06898/) (2021)
- [Balancing Interference and Correlation in Spatial Experimental Designs: A Causal Graph Cut Approach](https://mlsanity.com/paper/2505.20130/) (2025)
- [Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey](https://mlsanity.com/paper/2108.04462/) (2021)
- [SoK: Adversarial Machine Learning Attacks and Defences in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2301.04299/) (2023)
- [A Survey of Machine Learning-Based Ride-Hailing Planning](https://mlsanity.com/paper/2303.14646/) (2023)
- [Mutual Information as Intrinsic Reward of Reinforcement Learning Agents for On-demand Ride Pooling](https://mlsanity.com/paper/2312.15195/) (2023)
- [Multi-Agent Soft Actor-Critic with Coordinated Loss for Autonomous Mobility-on-Demand Fleet Control](https://mlsanity.com/paper/2404.06975/) (2024)
- [Coordinating Ride-Pooling with Public Transit using Reward-Guided Conservative Q-Learning: An Offline Training and Online Fine-Tuning Reinforcement Learning Framework](https://mlsanity.com/paper/2501.14199/) (2025)
- [Reinforcement learning for multi-item retrieval in the puzzle-based storage system](https://mlsanity.com/paper/2202.03424/) (2022)
- [Review of Metrics to Measure the Stability, Robustness and Resilience of Reinforcement Learning](https://mlsanity.com/paper/2203.12048/) (2022)
- [A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications](https://mlsanity.com/paper/2210.14524/) (2022)
- [Spatio-temporal Incentives Optimization for Ride-hailing Services with Offline Deep Reinforcement Learning](https://mlsanity.com/paper/2211.03240/) (2022)
- [Vehicle Dispatching and Routing of On-Demand Intercity Ride-Pooling Services: A Multi-Agent Hierarchical Reinforcement Learning Approach](https://mlsanity.com/paper/2307.06742/) (2023)
- [Deep reinforcement learning for the dynamic vehicle dispatching problem: An event-based approach](https://mlsanity.com/paper/2307.07508/) (2023)
- [A Unified Model for Spatio-Temporal Prediction Queries with Arbitrary Modifiable Areal Units](https://mlsanity.com/paper/2403.07022/) (2024)
- [Causal Deepsets for Off-policy Evaluation under Spatial or Spatio-temporal Interferences](https://mlsanity.com/paper/2407.17910/) (2024)
- [LLM-ODDR: A Large Language Model Framework for Joint Order Dispatching and Driver Repositioning](https://mlsanity.com/paper/2505.22695/) (2025)
- [A Two-armed Bandit Framework for A/B Testing](https://mlsanity.com/paper/2507.18118/) (2025)
- [DROP: Deep relocating option policy for optimal ride-hailing vehicle repositioning](https://mlsanity.com/paper/2109.04149/) (2021)

## Abstract (arXiv)

Recent works on ride-sharing order dispatching have highlighted the importance of taking into account both the spatial and temporal dynamics in the dispatching process for improving the transportation system efficiency. At the same time, deep reinforcement learning has advanced to the point where it achieves superhuman performance in a number of fields. In this work, we propose a deep reinforcement learning based solution for order dispatching and we conduct large scale online A/B tests on DiDi's ride-dispatching platform to show that the proposed method achieves significant improvement on both total driver income and user experience related metrics. In particular, we model the ride dispatching problem as a Semi Markov Decision Process to account for the temporal aspect of the dispatching actions. To improve the stability of the value iteration with nonlinear function approximators like neural networks, we propose Cerebellar Value Networks (CVNet) with a novel distributed state representation layer. We further derive a regularized policy evaluation scheme for CVNet that penalizes large Lipschitz constant of the value network for additional robustness against adversarial perturbation and noises. Finally, we adapt various transfer learning methods to CVNet for increased learning adaptability and efficiency across multiple cities. We conduct extensive offline simulations based on real dispatching data as well as online AB tests through the DiDi's platform. Results show that CVNet consistently outperforms other recently proposed dispatching methods. We finally show that the performance can be further improved through the efficient use of transfer learning.