# DRL4Route: A Deep Reinforcement Learning Framework for Pick-up and Delivery Route Prediction

arXiv: https://arxiv.org/abs/2307.16246 · HTML: https://mlsanity.com/paper/2307.16246/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 30 Jul 2023.

## Builds on (most cited first, 20 of 20)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Neural Combinatorial Optimization with Reinforcement Learning](https://mlsanity.com/paper/1611.09940/) (2016)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [A Deep Reinforced Model for Abstractive Summarization](https://mlsanity.com/paper/1705.04304/) (2017)
- [An Actor-Critic Algorithm for Sequence Prediction](https://mlsanity.com/paper/1607.07086/) (2016)
- [Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting](https://mlsanity.com/paper/1805.11080/) (2018)
- [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://mlsanity.com/paper/1802.08636/) (2018)
- [Improved Image Captioning via Policy Gradient optimization of SPIDEr](https://mlsanity.com/paper/1612.00370/) (2016)
- [Learning to Extract Coherent Summary via Deep Reinforcement Learning](https://mlsanity.com/paper/1804.07036/) (2018)
- [Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning](https://mlsanity.com/paper/1805.10627/) (2018)
- [Deep Reinforcement Learning For Sequence to Sequence Models](https://mlsanity.com/paper/1805.09461/) (2018)
- [A Reinforced Topic-Aware Convolutional Sequence-to-Sequence Model for Abstractive Text Summarization](https://mlsanity.com/paper/1805.03616/) (2018)
- [Differentiable Perturb-and-Parse: Semi-Supervised Parsing with a Structured Variational Autoencoder](https://mlsanity.com/paper/1807.09875/) (2018)
- [Cold-Start Reinforcement Learning with Softmax Policy Gradient](https://mlsanity.com/paper/1709.09346/) (2017)
- [Learning to Route with Sparse Trajectory Sets---Extended Version](https://mlsanity.com/paper/1802.07980/) (2018)

## Built on by (most cited first, 10 of 10)

- [DutyTTE: Deciphering Uncertainty in Origin-Destination Travel Time Estimation](https://mlsanity.com/paper/2408.12809/) (2024)
- [A Survey on Service Route and Time Prediction in Instant Delivery: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2309.01194/) (2023)
- [Learning to Estimate Package Delivery Time in Mixed Imbalanced Delivery and Pickup Logistics Services](https://mlsanity.com/paper/2505.00375/) (2025)
- [MRGRP: Empowering Courier Route Prediction in Food Delivery Service with Multi-Relational Graph](https://mlsanity.com/paper/2505.11999/) (2025)
- [Towards An Efficient and Effective En Route Travel Time Estimation Framework](https://mlsanity.com/paper/2504.04086/) (2025)
- [DRL4AOI: A DRL Framework for Semantic-aware AOI Segmentation in Location-Based Services](https://mlsanity.com/paper/2412.05437/) (2024)
- [PAPN: Proximity Attention Encoder and Pointer Network Decoder for Parcel Pickup Route Prediction](https://mlsanity.com/paper/2505.03776/) (2025)
- [A Coalition Game for On-demand Multi-modal 3D Automated Delivery System](https://mlsanity.com/paper/2412.17252/) (2024)
- [Weather-Aware Transformer for Real-Time Route Optimization in Drone-as-a-Service Operations](https://mlsanity.com/paper/2601.03376/) (2026)
- [ORBITER: Conflict-Aware Decision-Making for Agentic Last-Mile Delivery](https://mlsanity.com/paper/2608.18846/) (2026)

## Abstract (arXiv)

Pick-up and Delivery Route Prediction (PDRP), which aims to estimate the future service route of a worker given his current task pool, has received rising attention in recent years. Deep neural networks based on supervised learning have emerged as the dominant model for the task because of their powerful ability to capture workers' behavior patterns from massive historical data. Though promising, they fail to introduce the non-differentiable test criteria into the training process, leading to a mismatch in training and test criteria. Which considerably trims down their performance when applied in practical systems. To tackle the above issue, we present the first attempt to generalize Reinforcement Learning (RL) to the route prediction task, leading to a novel RL-based framework called DRL4Route. It combines the behavior-learning abilities of previous deep learning models with the non-differentiable objective optimization ability of reinforcement learning. DRL4Route can serve as a plug-and-play component to boost the existing deep learning models. Based on the framework, we further implement a model named DRL4Route-GAE for PDRP in logistic service. It follows the actor-critic architecture which is equipped with a Generalized Advantage Estimator that can balance the bias and variance of the policy gradient estimates, thus achieving a more optimal policy. Extensive offline experiments and the online deployment show that DRL4Route-GAE improves Location Square Deviation (LSD) by 0.9%-2.7%, and Accuracy@3 (ACC@3) by 2.4%-3.2% over existing methods on the real-world dataset.