# Deep Q-Learning for Same-Day Delivery with Vehicles and Drones

arXiv: https://arxiv.org/abs/1910.11901 · HTML: https://mlsanity.com/paper/1910.11901/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Oct 2019.

## Built on by (most cited first, 15 of 15)

- [Learning to Solve Vehicle Routing Problems: A Survey](https://mlsanity.com/paper/2205.02453/) (2022)
- [Learning for routing: A guided review of recent developments and future directions](https://mlsanity.com/paper/2507.00218/) (2025)
- [A Deep Reinforcement Learning Approach for the Meal Delivery Problem](https://mlsanity.com/paper/2104.12000/) (2021)
- [Deep Reinforcement Learning with Explicit Context Representation](https://mlsanity.com/paper/2310.09924/) (2023)
- [Reinforcement Learning Approaches for the Orienteering Problem with Stochastic and Dynamic Release Dates](https://mlsanity.com/paper/2207.00885/) (2022)
- [Off-line approximate dynamic programming for the vehicle routing problem with a highly variable customer basis and stochastic demands](https://mlsanity.com/paper/2109.10200/) (2021)
- [Neural Approximate Dynamic Programming for the Ultra-fast Order Dispatching Problem](https://mlsanity.com/paper/2311.12975/) (2023)
- [A Markov Decision Process Approach for Managing Medical Drone Deliveries](https://mlsanity.com/paper/2106.04729/) (2021)
- [Enhancing Courier Scheduling in Crowdsourced Last-Mile Delivery through Dynamic Shift Extensions: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2402.09961/) (2024)
- [Constrained Reinforcement Learning for the Dynamic Inventory Routing Problem under Stochastic Supply and Demand](https://mlsanity.com/paper/2503.05276/) (2025)
- [To Start Up a Start-Up-Embedding Strategic Demand Development in Operational On-Demand Fulfillment via Reinforcement Learning with Information Shaping](https://mlsanity.com/paper/2504.05633/) (2025)
- [Deep Reinforcement Learning for Drone Route Optimization in Post-Disaster Road Assessment](https://mlsanity.com/paper/2509.01886/) (2025)
- [Demand Acceptance using Reinforcement Learning for Dynamic Vehicle Routing Problem with Emission Quota](https://mlsanity.com/paper/2603.13279/) (2026)
- [A Deep Reinforcement Learning Algorithm for the Vehicle Routing Problem with Stochastic Demands and Outsourcing](https://mlsanity.com/paper/2607.16875/) (2026)
- [Reinforcement Learning in Operational Research: A Technical Review and Practical Roadmap](https://mlsanity.com/paper/2609.24750/) (2026)

## Abstract (arXiv)

In this paper, we consider same-day delivery with vehicles and drones. Customers make delivery requests over the course of the day, and the dispatcher dynamically dispatches vehicles and drones to deliver the goods to customers before their delivery deadline. Vehicles can deliver multiple packages in one route but travel relatively slowly due to the urban traffic. Drones travel faster, but they have limited capacity and require charging or battery swaps. To exploit the different strengths of the fleets, we propose a deep Q-learning approach. Our method learns the value of assigning a new customer to either drones or vehicles as well as the option to not offer service at all. In a systematic computational analysis, we show the superiority of our policy compared to benchmark policies and the effectiveness of our deep Q-learning approach. We also show that our policy can maintain effectiveness when the fleet size changes moderately. Experiments on data drawn from varied spatial/temporal distributions demonstrate that our trained policies can cope with changes in the input data.