# Learning Practically Feasible Policies for Online 3D Bin Packing

arXiv: https://arxiv.org/abs/2108.13680 · HTML: https://mlsanity.com/paper/2108.13680/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 51 references that are in the map.
- First posted on arXiv 31 Aug 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Attention, Learn to Solve Routing Problems!](https://mlsanity.com/paper/1803.08475/) (2018)
- [Neural Combinatorial Optimization with Reinforcement Learning](https://mlsanity.com/paper/1611.09940/) (2016)
- [Risk-Constrained Reinforcement Learning with Percentile Risk Criteria](https://mlsanity.com/paper/1512.01629/) (2015)
- [Distributed Distributional Deterministic Policy Gradients](https://mlsanity.com/paper/1804.08617/) (2018)
- [Learning to Dispatch for Job Shop Scheduling via Deep Reinforcement Learning](https://mlsanity.com/paper/2010.12367/) (2020)
- [Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation](https://mlsanity.com/paper/1708.05144/) (2017)
- [Action Branching Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1711.08946/) (2017)
- [Online 3D Bin Packing with Constrained Deep Reinforcement Learning](https://mlsanity.com/paper/2006.14978/) (2020)
- [Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method](https://mlsanity.com/paper/1708.05930/) (2017)
- [Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimization](https://mlsanity.com/paper/1807.01672/) (2018)
- [Optimal least-squares solution to the hand-eye calibration problem](https://mlsanity.com/paper/2002.10838/) (2020)

## Built on by (most cited first, 18 of 18)

- [Learning Physically Realizable Skills for Online Packing of General 3D Shapes](https://mlsanity.com/paper/2212.02094/) (2022)
- [DISCO: Efficient Diffusion Solver for Large-Scale Combinatorial Optimization Problems](https://mlsanity.com/paper/2406.19705/) (2024)
- [Adjustable Robust Reinforcement Learning for Online 3D Bin Packing](https://mlsanity.com/paper/2310.04323/) (2023)
- [Learning based 2D Irregular Shape Packing](https://mlsanity.com/paper/2309.10329/) (2023)
- [Machine Learning for the Multi-Dimensional Bin Packing Problem: Literature Review and Empirical Evaluation](https://mlsanity.com/paper/2312.08103/) (2023)
- [GOPT: Generalizable Online 3D Bin Packing via Transformer-based Deep Reinforcement Learning](https://mlsanity.com/paper/2409.05344/) (2024)
- [Deliberate Planning of 3D Bin Packing on Packing Configuration Trees](https://mlsanity.com/paper/2504.04421/) (2025)
- [Learning Gradient Fields for Scalable and Generalizable Irregular Packing](https://mlsanity.com/paper/2310.19814/) (2023)
- [Efficient Reinforcement Learning of Task Planners for Robotic Palletization through Iterative Action Masking Learning](https://mlsanity.com/paper/2404.04772/) (2024)
- [Learning Dual-arm Object Rearrangement for Cartesian Robots](https://mlsanity.com/paper/2402.13634/) (2024)
- [A Hierarchical Bin Packing Framework with Dual Manipulators via Heuristic Search and Deep Reinforcement Learning](https://mlsanity.com/paper/2506.01628/) (2025)
- [A comparative analysis of deep learning models for lung segmentation on X-ray images](https://mlsanity.com/paper/2404.06455/) (2024)
- [Mitigating Dimensionality in 2D Rectangle Packing Problem under Reinforcement Learning Schema](https://mlsanity.com/paper/2409.09677/) (2024)
- [Cartesian Genetic Programming Approach for Designing Convolutional Neural Networks](https://mlsanity.com/paper/2410.00129/) (2024)
- [Optimizing 2D+1 Packing in Constrained Environments Using Deep Reinforcement Learning](https://mlsanity.com/paper/2503.17573/) (2025)
- [One4Many-StablePacker: An Efficient Deep Reinforcement Learning Framework for the 3D Bin Packing Problem](https://mlsanity.com/paper/2510.10057/) (2025)
- [Effective Online 3D Bin Packing with Lookahead Parcels Using Monte Carlo Tree Search](https://mlsanity.com/paper/2601.02649/) (2026)
- [Learning Flexible Job Shop Scheduling under Limited Buffers and Material Kitting Constraints](https://mlsanity.com/paper/2602.24180/) (2026)

## Abstract (arXiv)

We tackle the Online 3D Bin Packing Problem, a challenging yet practically useful variant of the classical Bin Packing Problem. In this problem, the items are delivered to the agent without informing the full sequence information. Agent must directly pack these items into the target bin stably without changing their arrival order, and no further adjustment is permitted. Online 3D-BPP can be naturally formulated as Markov Decision Process (MDP). We adopt deep reinforcement learning, in particular, the on-policy actor-critic framework, to solve this MDP with constrained action space. To learn a practically feasible packing policy, we propose three critical designs. First, we propose an online analysis of packing stability based on a novel stacking tree. It attains a high analysis accuracy while reducing the computational complexity from O(N2) to O(N log N), making it especially suited for RL training. Second, we propose a decoupled packing policy learning for different dimensions of placement which enables high-resolution spatial discretization and hence high packing precision. Third, we introduce a reward function that dictates the robot to place items in a far-to-near order and therefore simplifies the collision avoidance in movement planning of the robotic arm. Furthermore, we provide a comprehensive discussion on several key implemental issues. The extensive evaluation demonstrates that our learned policy outperforms the state-of-the-art methods significantly and is practically usable for real-world applications.