# Effective Online 3D Bin Packing with Lookahead Parcels Using Monte Carlo Tree Search

arXiv: https://arxiv.org/abs/2601.02649 · HTML: https://mlsanity.com/paper/2601.02649/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 49 references that are in the map.
- First posted on arXiv 6 Jan 2026.

## Builds on (most cited first, 14 of 14)

- [Monte Carlo Tree Search: A Review of Recent Modifications and Applications](https://mlsanity.com/paper/2103.04931/) (2021)
- [Robust Reinforcement Learning using Offline Data](https://mlsanity.com/paper/2208.05129/) (2022)
- [Online 3D Bin Packing with Constrained Deep Reinforcement Learning](https://mlsanity.com/paper/2006.14978/) (2020)
- [Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method](https://mlsanity.com/paper/1708.05930/) (2017)
- [Stable bin packing of non-convex 3D objects with a robot manipulator](https://mlsanity.com/paper/1812.04093/) (2018)
- [Learning Practically Feasible Policies for Online 3D Bin Packing](https://mlsanity.com/paper/2108.13680/) (2021)
- [How to Combine Tree-Search Methods in Reinforcement Learning](https://mlsanity.com/paper/1809.01843/) (2018)
- [TAP-Net: Transport-and-Pack using Reinforcement Learning](https://mlsanity.com/paper/2009.01469/) (2020)
- [A Generalized Reinforcement Learning Algorithm for Online 3D Bin-Packing](https://mlsanity.com/paper/2007.00463/) (2020)
- [Adjustable Robust Reinforcement Learning for Online 3D Bin Packing](https://mlsanity.com/paper/2310.04323/) (2023)
- [Online 3D Bin Packing Reinforcement Learning Solution with Buffer](https://mlsanity.com/paper/2208.07123/) (2022)
- [The Value of Reward Lookahead in Reinforcement Learning](https://mlsanity.com/paper/2403.11637/) (2024)
- [Think Too Fast Nor Too Slow: The Computational Trade-off Between Planning And Reinforcement Learning](https://mlsanity.com/paper/2005.07404/) (2020)
- [A generalized algorithm and framework for online 3-dimensional bin packing in an automated sorting center](https://mlsanity.com/paper/2111.01072/) (2021)

## Abstract (arXiv)

Online 3D Bin Packing (3D-BP) with robotic arms is crucial for reducing transportation and labor costs in modern logistics. While Deep Reinforcement Learning (DRL) has shown strong performance, it often fails to adapt to real-world short-term distribution shifts, which arise as different batches of goods arrive sequentially, causing performance drops. We argue that the short-term lookahead information available in modern logistics systems is key to mitigating this issue, especially during distribution shifts. We formulate online 3D-BP with lookahead parcels as a Model Predictive Control (MPC) problem and adapt the Monte Carlo Tree Search (MCTS) framework to solve it. Our framework employs a dynamic exploration prior that automatically balances a learned RL policy and a robust random policy based on the lookahead characteristics. Additionally, we design an auxiliary reward to penalize long-term spatial waste from individual placements. Extensive experiments on real-world datasets show that our method consistently outperforms state-of-the-art baselines, achieving over 10% gains under distributional shifts, 4% average improvement in online deployment, and up to more than 8% in the best case--demonstrating the effectiveness of our framework.