# Online 3D Bin Packing Reinforcement Learning Solution with Buffer

arXiv: https://arxiv.org/abs/2208.07123 · HTML: https://mlsanity.com/paper/2208.07123/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Builds on (most cited first, 14 of 14)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Neural Combinatorial Optimization with Reinforcement Learning](https://mlsanity.com/paper/1611.09940/) (2016)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [Imagination-Augmented Agents for Deep Reinforcement Learning](https://mlsanity.com/paper/1707.06203/) (2017)
- [Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation](https://mlsanity.com/paper/1708.05144/) (2017)
- [Online 3D Bin Packing with Constrained Deep Reinforcement Learning](https://mlsanity.com/paper/2006.14978/) (2020)
- [Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method](https://mlsanity.com/paper/1708.05930/) (2017)
- [Train on Small, Play the Large: Scaling Up Board Games with AlphaZero and GNN](https://mlsanity.com/paper/2107.08387/) (2021)

## Built on by (most cited first, 8 of 8)

- [Machine Learning for the Multi-Dimensional Bin Packing Problem: Literature Review and Empirical Evaluation](https://mlsanity.com/paper/2312.08103/) (2023)
- [Deliberate Planning of 3D Bin Packing on Packing Configuration Trees](https://mlsanity.com/paper/2504.04421/) (2025)
- [A comparative analysis of deep learning models for lung segmentation on X-ray images](https://mlsanity.com/paper/2404.06455/) (2024)
- [Mitigating Dimensionality in 2D Rectangle Packing Problem under Reinforcement Learning Schema](https://mlsanity.com/paper/2409.09677/) (2024)
- [Cartesian Genetic Programming Approach for Designing Convolutional Neural Networks](https://mlsanity.com/paper/2410.00129/) (2024)
- [Optimizing 2D+1 Packing in Constrained Environments Using Deep Reinforcement Learning](https://mlsanity.com/paper/2503.17573/) (2025)
- [OPA-Pack: Object-Property-Aware Robotic Bin Packing](https://mlsanity.com/paper/2505.13339/) (2025)
- [Effective Online 3D Bin Packing with Lookahead Parcels Using Monte Carlo Tree Search](https://mlsanity.com/paper/2601.02649/) (2026)

## Abstract (arXiv)

The 3D Bin Packing Problem (3D-BPP) is one of the most demanded yet challenging problems in industry, where an agent must pack variable size items delivered in sequence into a finite bin with the aim to maximize the space utilization. It represents a strongly NP-Hard optimization problem such that no solution has been offered to date with high performance in space utilization. In this paper, we present a new reinforcement learning (RL) framework for a 3D-BPP solution for improving performance. First, a buffer is introduced to allow multi-item action selection. By increasing the degree of freedom in action selection, a more complex policy that results in better packing performance can be derived. Second, we propose an agnostic data augmentation strategy that exploits both bin item symmetries for improving sample efficiency. Third, we implement a model-based RL method adapted from the popular algorithm AlphaGo, which has shown superhuman performance in zero-sum games. Our adaptation is capable of working in single-player and score based environments. In spite of the fact that AlphaGo versions are known to be computationally heavy, we manage to train the proposed framework with a single thread and GPU, while obtaining a solution that outperforms the state-of-the-art results in space utilization.