# Value Function is All You Need: A Unified Learning Framework for Ride Hailing Platforms

arXiv: https://arxiv.org/abs/2105.08791 · HTML: https://mlsanity.com/paper/2105.08791/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 24 references that are in the map.
- First posted on arXiv 18 May 2021.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [MOVI: A Model-Free Approach to Dynamic Fleet Management](https://mlsanity.com/paper/1804.04758/) (2018)
- [Optimal Passenger-Seeking Policies on E-hailing Platforms Using Markov Decision Process and Imitation Learning](https://mlsanity.com/paper/1905.09906/) (2019)

## Built on by (most cited first, 16 of 16)

- [Reinforcement Learning in the Wild: Scalable RL Dispatching Algorithm Deployed in Ridehailing Marketplace](https://mlsanity.com/paper/2202.05118/) (2022)
- [An End-to-End Reinforcement Learning Based Approach for Micro-View Order-Dispatching in Ride-Hailing](https://mlsanity.com/paper/2408.10479/) (2024)
- [Deep Reinforcement Learning for Demand Driven Services in Logistics and Transportation Systems: A Survey](https://mlsanity.com/paper/2108.04462/) (2021)
- [Interpretable Personalized Experimentation](https://mlsanity.com/paper/2111.03267/) (2021)
- [A Survey of Machine Learning-Based Ride-Hailing Planning](https://mlsanity.com/paper/2303.14646/) (2023)
- [Coordinating Ride-Pooling with Public Transit using Reward-Guided Conservative Q-Learning: An Offline Training and Online Fine-Tuning Reinforcement Learning Framework](https://mlsanity.com/paper/2501.14199/) (2025)
- [Learning to Control Autonomous Fleets from Observation via Offline Reinforcement Learning](https://mlsanity.com/paper/2302.14833/) (2023)
- [Deep reinforcement learning for the dynamic vehicle dispatching problem: An event-based approach](https://mlsanity.com/paper/2307.07508/) (2023)
- [LLM-ODDR: A Large Language Model Framework for Joint Order Dispatching and Driver Repositioning](https://mlsanity.com/paper/2505.22695/) (2025)
- [A Unified Representation Framework for Rideshare Marketplace Equilibrium and Efficiency](https://mlsanity.com/paper/2302.14358/) (2023)
- [Online Relocating and Matching of Ride-Hailing Services: A Model-Based Modular Approach](https://mlsanity.com/paper/2310.09071/) (2023)
- [ProfiLLM: Utility-Aligned Agentic User Profiling for Industrial Ride-Hailing Dispatch](https://mlsanity.com/paper/2606.18803/) (2026)
- [Non-myopic Matching and Rebalancing in Large-Scale On-Demand Ride-Pooling Systems Using Simulation-Informed Reinforcement Learning](https://mlsanity.com/paper/2510.25796/) (2025)
- [Partially Observable Learning for Multi-Platform Dispatch Optimization](https://mlsanity.com/paper/2608.10897/) (2026)
- [GenMatch: An End-to-End Generative Matching Framework for Micro-View Order-Dispatching in Ride-Hailing](https://mlsanity.com/paper/2608.19751/) (2026)
- [JointMatch: A Unified Heterogeneous Graph Neural Solver for Large-Scale Ride-Sharing Matching](https://mlsanity.com/paper/2609.20200/) (2026)

## Abstract (arXiv)

Large ride-hailing platforms, such as DiDi, Uber and Lyft, connect tens of thousands of vehicles in a city to millions of ride demands throughout the day, providing great promises for improving transportation efficiency through the tasks of order dispatching and vehicle repositioning. Existing studies, however, usually consider the two tasks in simplified settings that hardly address the complex interactions between the two, the real-time fluctuations between supply and demand, and the necessary coordinations due to the large-scale nature of the problem. In this paper we propose a unified value-based dynamic learning framework (V1D3) for tackling both tasks. At the center of the framework is a globally shared value function that is updated continuously using online experiences generated from real-time platform transactions. To improve the sample-efficiency and the robustness, we further propose a novel periodic ensemble method combining the fast online learning with a large-scale offline training scheme that leverages the abundant historical driver trajectory data. This allows the proposed framework to adapt quickly to the highly dynamic environment, to generalize robustly to recurrent patterns and to drive implicit coordinations among the population of managed vehicles. Extensive experiments based on real-world datasets show considerably improvements over other recently proposed methods on both tasks. Particularly, V1D3 outperforms the first prize winners of both dispatching and repositioning tracks in the KDD Cup 2020 RL competition, achieving state-of-the-art results on improving both total driver income and user experience related metrics.