# Online 3D Bin Packing with Constrained Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2006.14978 · HTML: https://mlsanity.com/paper/2006.14978/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2020.

## Built on by (most cited first, 30 of 41)

- [Learning to Search Feasible and Infeasible Regions of Routing Problems with Flexible Neural k-Opt](https://mlsanity.com/paper/2310.18264/) (2023)
- [Learning to Handle Complex Constraints for Vehicle Routing Problems](https://mlsanity.com/paper/2410.21066/) (2024)
- [Meta-Learning-Based Deep Reinforcement Learning for Multiobjective Optimization Problems](https://mlsanity.com/paper/2105.02741/) (2021)
- [Learning Practically Feasible Policies for Online 3D Bin Packing](https://mlsanity.com/paper/2108.13680/) (2021)
- [Attend2Pack: Bin Packing through Deep Reinforcement Learning with Attention](https://mlsanity.com/paper/2107.04333/) (2021)
- [Learning Physically Realizable Skills for Online Packing of General 3D Shapes](https://mlsanity.com/paper/2212.02094/) (2022)
- [Online Bin Packing with Predictions](https://mlsanity.com/paper/2102.03311/) (2021)
- [DISCO: Efficient Diffusion Solver for Large-Scale Combinatorial Optimization Problems](https://mlsanity.com/paper/2406.19705/) (2024)
- [Adjustable Robust Reinforcement Learning for Online 3D Bin Packing](https://mlsanity.com/paper/2310.04323/) (2023)
- [Online 3D Bin Packing Reinforcement Learning Solution with Buffer](https://mlsanity.com/paper/2208.07123/) (2022)
- [Planning Irregular Object Packing via Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2211.09382/) (2022)
- [Learning to Design and Construct Bridge without Blueprint](https://mlsanity.com/paper/2108.02439/) (2021)
- [WISK: A Workload-aware Learned Index for Spatial Keyword Queries](https://mlsanity.com/paper/2302.14287/) (2023)
- [(X)-evolve: Solution space evolution powered by large language models](https://mlsanity.com/paper/2508.07932/) (2025)
- [Machine Learning for the Multi-Dimensional Bin Packing Problem: Literature Review and Empirical Evaluation](https://mlsanity.com/paper/2312.08103/) (2023)
- [Bin Packing Optimization via Deep Reinforcement Learning](https://mlsanity.com/paper/2403.12420/) (2024)
- [GOPT: Generalizable Online 3D Bin Packing via Transformer-based Deep Reinforcement Learning](https://mlsanity.com/paper/2409.05344/) (2024)
- [Deliberate Planning of 3D Bin Packing on Packing Configuration Trees](https://mlsanity.com/paper/2504.04421/) (2025)
- [Neural Packing: from Visual Sensing to Reinforcement Learning](https://mlsanity.com/paper/2311.09233/) (2023)
- [Efficient Reinforcement Learning of Task Planners for Robotic Palletization through Iterative Action Masking Learning](https://mlsanity.com/paper/2404.04772/) (2024)
- [FloorSet -- a VLSI Floorplanning Dataset with Design Constraints of Real-World SoCs](https://mlsanity.com/paper/2405.05480/) (2024)
- [GFPack++: Improving 2D Irregular Packing by Learning Gradient Field with Attention](https://mlsanity.com/paper/2406.07579/) (2024)
- [An Efficient Deep Reinforcement Learning Model for Online 3D Bin Packing Combining Object Rearrangement and Stable Placement](https://mlsanity.com/paper/2408.09694/) (2024)
- [Learning Dual-arm Object Rearrangement for Cartesian Robots](https://mlsanity.com/paper/2402.13634/) (2024)
- [ASAP: Exploiting the Satisficing Generalization Edge in Neural Combinatorial Optimization](https://mlsanity.com/paper/2501.17377/) (2025)
- [Learning Virtual Machine Scheduling in Cloud Computing through Language Agents](https://mlsanity.com/paper/2505.10117/) (2025)
- [A Hierarchical Bin Packing Framework with Dual Manipulators via Heuristic Search and Deep Reinforcement Learning](https://mlsanity.com/paper/2506.01628/) (2025)
- [A Virtual-Force Based Swarm Algorithm for Balanced Circular Bin Packing Problems](https://mlsanity.com/paper/2306.01021/) (2023)
- [Using Reinforcement Learning for the Three-Dimensional Loading Capacitated Vehicle Routing Problem](https://mlsanity.com/paper/2307.12136/) (2023)
- [A comparative analysis of deep learning models for lung segmentation on X-ray images](https://mlsanity.com/paper/2404.06455/) (2024)

## Abstract (arXiv)

We solve a challenging yet practically useful variant of 3D Bin Packing Problem (3D-BPP). In our problem, the agent has limited information about the items to be packed into the bin, and an item must be packed immediately after its arrival without buffering or readjusting. The item's placement also subjects to the constraints of collision avoidance and physical stability. We formulate this online 3D-BPP as a constrained Markov decision process. To solve the problem, we propose an effective and easy-to-implement constrained deep reinforcement learning (DRL) method under the actor-critic framework. In particular, we introduce a feasibility predictor to predict the feasibility mask for the placement actions and use it to modulate the action probabilities output by the actor during training. Such supervisions and transformations to DRL facilitate the agent to learn feasible policies efficiently. Our method can also be generalized e.g., with the ability to handle lookahead or items with different orientations. We have conducted extensive evaluation showing that the learned policy significantly outperforms the state-of-the-art methods. A user study suggests that our method attains a human-level performance.