# Solving the Order Batching and Sequencing Problem using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2006.09507 · HTML: https://mlsanity.com/paper/2006.09507/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2020.

## Built on by (most cited first, 5 of 5)

- [Deep Reinforcement Learning for Dynamic Order Picking in Warehouse Operations](https://mlsanity.com/paper/2408.01656/) (2024)
- [A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning](https://mlsanity.com/paper/2208.00808/) (2022)
- [Topology-Aware and Highly Generalizable Deep Reinforcement Learning for Efficient Retrieval in Multi-Deep Storage Systems](https://mlsanity.com/paper/2506.14787/) (2025)
- [Quantum-Inspired DRL Approach with LSTM and OU Noise for Cut Order Planning Optimization](https://mlsanity.com/paper/2508.16611/) (2025)
- [Offline Reinforcement Learning for Warehouse SLAM Throughput Control](https://mlsanity.com/paper/2606.23978/) (2026)

## Abstract (arXiv)

In e-commerce markets, on time delivery is of great importance to customer satisfaction. In this paper, we present a Deep Reinforcement Learning (DRL) approach for deciding how and when orders should be batched and picked in a warehouse to minimize the number of tardy orders. In particular, the technique facilitates making decisions on whether an order should be picked individually (pick-by-order) or picked in a batch with other orders (pick-by-batch), and if so with which other orders. We approach the problem by formulating it as a semi-Markov decision process and develop a vector-based state representation that includes the characteristics of the warehouse system. This allows us to create a deep reinforcement learning solution that learns a strategy by interacting with the environment and solve the problem with a proximal policy optimization algorithm. We evaluate the performance of the proposed DRL approach by comparing it with several batching and sequencing heuristics in different problem settings. The results show that the DRL approach is able to develop a strategy that produces consistent, good solutions and performs better than the proposed heuristics.