Multi-Agent Reinforcement Learning from Delayed Marketplace Feedback for Objective-Weight Adaptation in Three-Sided Dispatch
Not on the map. Why?
Abstract (arXiv)
Dispatch in three-sided marketplaces provides a natural setting for reinforcement learning from world feedback: decisions are evaluated by delayed operational outcomes such as delivery speed, courier utilization, and merchant congestion. We present a deployed reinforcement learning system at DoorDash that adapts dispatch objective weights in a large-scale food-delivery marketplace using delayed signals. Rather than replacing the combinatorial assignment optimizer, a store-level policy learned from logged marketplace data selects a discrete multiplier that shifts the dispatch optimizer's tradeoff between delivery quality and batching efficiency. This interface enables offline policy learning under noisy, delayed, and coupled feedback while preserving production feasibility constraints and operational safeguards. We train a shared value function using centralized offline data and decentralized store-level execution, with Double Q-learning targets and a conservative regularizer to reduce out-of-distribution value overestimation. In a production switchback experiment, the offline-trained policy increases batching and reduces courier-side time costs without degrading customer-facing delivery quality. Results illustrate how world feedback from a live economic and logistics system can be used to safely adapt decision policies online.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 12 references that are in the map. First posted on arXiv 11 Jun 2026.
Builds on
5 of 12 references.
References · 5
- 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
- 2020Design and Analysis of Switchback Experimentscited by 38
- 2021A Deep Reinforcement Learning Approach for the Meal Delivery Problemcited by 5
- 2025Real-Time Integrated Dispatching and Idle Fleet Steering with Deep Reinforcement Learning for A Meal Delivery Platformcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Partially Observable Learning for Multi-Platform Dispatch Optimization
- 2024Multi-agent Reinforcement Learning for Dynamic Dispatching in Material Handling Systems
- 2024Reinforcement Learning as an Improvement Heuristic for Real-World Production Scheduling
- 2025GRAND: Guidance, Rebalancing, and Assignment for Networked Dispatch in Multi-Agent Path Finding
- 2024Offline Reinforcement Learning for Learning to Dispatch for Job Shop Scheduling
- 2025OpComm: A Reinforcement Learning Framework for Adaptive Buffer Control in Warehouse Volume Forecasting
- 2025One Step is Enough: Multi-Agent Reinforcement Learning based on One-Step Policy Optimization for Order Dispatch on Ride-Sharing Platforms
- 2026An Analysis of the Coordination Gap between Joint and Modular Learning for Job Shop Scheduling with Transportation Resources