# An End-to-End Reinforcement Learning Based Approach for Micro-View Order-Dispatching in Ride-Hailing

arXiv: https://arxiv.org/abs/2408.10479 · HTML: https://mlsanity.com/paper/2408.10479/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 20 Aug 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [PixelSNAIL: An Improved Autoregressive Generative Model](https://mlsanity.com/paper/1712.09763/) (2017)
- [On Empirical Comparisons of Optimizers for Deep Learning](https://mlsanity.com/paper/1910.05446/) (2019)
- [Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1901.11454/) (2019)
- [Discrete Sequential Prediction of Continuous Actions for Deep RL](https://mlsanity.com/paper/1705.05035/) (2017)
- [A Deep Value-network Based Approach for Multi-Driver Order Dispatching](https://mlsanity.com/paper/2106.04493/) (2021)
- [Reinforcement Learning for Ridesharing: An Extended Survey](https://mlsanity.com/paper/2105.01099/) (2021)
- [Deep Reinforcement Learning for Multi-Driver Vehicle Dispatching and Repositioning Problem](https://mlsanity.com/paper/1911.11260/) (2019)
- [Multi-Agent Reinforcement Learning for Order-dispatching via Order-Vehicle Distribution Matching](https://mlsanity.com/paper/1910.02591/) (2019)
- [CoRide: Joint Order Dispatching and Fleet Management for Multi-Scale Ride-Hailing Platforms](https://mlsanity.com/paper/1905.11353/) (2019)
- [Value Function is All You Need: A Unified Learning Framework for Ride Hailing Platforms](https://mlsanity.com/paper/2105.08791/) (2021)
- [Dynamic Type Matching](https://mlsanity.com/paper/1811.07048/) (2018)
- [A Deep Reinforcement Learning Approach for Online Parcel Assignment](https://mlsanity.com/paper/2109.03467/) (2021)

## Built on by (most cited first, 7 of 7)

- [Hierarchical Optimization via LLM-Guided Objective Evolution for Mobility-on-Demand Systems](https://mlsanity.com/paper/2510.10644/) (2025)
- [ProfiLLM: Utility-Aligned Agentic User Profiling for Industrial Ride-Hailing Dispatch](https://mlsanity.com/paper/2606.18803/) (2026)
- [Optimizing Drivers' Discount Order Acceptance Strategies: A Policy-Improved Deep Deterministic Policy Gradient Framework](https://mlsanity.com/paper/2507.11865/) (2025)
- [Adaptive Value Decomposition: Coordinating a Varying Number of Agents in Urban Systems](https://mlsanity.com/paper/2602.13309/) (2026)
- [A Distributionally Robust Reinforcement Learning Framework for Constrained Urban EV Dispatch](https://mlsanity.com/paper/2604.25848/) (2026)
- [GenMatch: An End-to-End Generative Matching Framework for Micro-View Order-Dispatching in Ride-Hailing](https://mlsanity.com/paper/2608.19751/) (2026)
- [RideSkill: A Hierarchical Algorithm for Generalized Ride Sharing with LLM-Driven Automatic Evolution](https://mlsanity.com/paper/2609.02250/) (2026)

## Abstract (arXiv)

Assigning orders to drivers under localized spatiotemporal context (micro-view order-dispatching) is a major task in Didi, as it influences ride-hailing service experience. Existing industrial solutions mainly follow a two-stage pattern that incorporate heuristic or learning-based algorithms with naive combinatorial methods, tackling the uncertainty of both sides' behaviors, including emerging timings, spatial relationships, and travel duration, etc. In this paper, we propose a one-stage end-to-end reinforcement learning based order-dispatching approach that solves behavior prediction and combinatorial optimization uniformly in a sequential decision-making manner. Specifically, we employ a two-layer Markov Decision Process framework to model this problem, and present underlineDeep underlineDouble underlineScalable underlineNetwork (D2SN), an encoder-decoder structure network to generate order-driver assignments directly and stop assignments accordingly. Besides, by leveraging contextual dynamics, our approach can adapt to the behavioral patterns for better performance. Extensive experiments on Didi's real-world benchmarks justify that the proposed approach significantly outperforms competitive baselines in optimizing matching efficiency and user experience tasks. In addition, we evaluate the deployment outline and discuss the gains and experiences obtained during the deployment tests from the view of large-scale engineering implementation.