# Planning Irregular Object Packing via Hierarchical Reinforcement Learning

arXiv: https://arxiv.org/abs/2211.09382 · HTML: https://mlsanity.com/paper/2211.09382/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 17 Nov 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [FeUdal Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1703.01161/) (2017)
- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://mlsanity.com/paper/1604.06057/) (2016)
- [Reinforcement Learning for Combinatorial Optimization: A Survey](https://mlsanity.com/paper/2003.03600/) (2020)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [Stochastic Neural Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1704.03012/) (2017)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [SGM: Sequence Generation Model for Multi-label Classification](https://mlsanity.com/paper/1806.04822/) (2018)
- [Video Captioning via Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1711.11135/) (2017)
- [Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2110.13625/) (2021)
- [Online 3D Bin Packing with Constrained Deep Reinforcement Learning](https://mlsanity.com/paper/2006.14978/) (2020)
- [Towards mental time travel: a hierarchical memory for reinforcement learning agents](https://mlsanity.com/paper/2105.14039/) (2021)
- [Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method](https://mlsanity.com/paper/1708.05930/) (2017)
- [OCRTOC: A Cloud-Based Competition and Benchmark for Robotic Grasping and Manipulation](https://mlsanity.com/paper/2104.11446/) (2021)
- [Stable bin packing of non-convex 3D objects with a robot manipulator](https://mlsanity.com/paper/1812.04093/) (2018)
- [Intelligent problem-solving as integrated hierarchical reinforcement learning](https://mlsanity.com/paper/2208.08731/) (2022)
- [Attend2Pack: Bin Packing through Deep Reinforcement Learning with Attention](https://mlsanity.com/paper/2107.04333/) (2021)
- [TAP-Net: Transport-and-Pack using Reinforcement Learning](https://mlsanity.com/paper/2009.01469/) (2020)
- [PackIt: A Virtual Environment for Geometric Planning](https://mlsanity.com/paper/2007.11121/) (2020)
- [GE-Grasp: Efficient Target-Oriented Grasping in Dense Clutter](https://mlsanity.com/paper/2207.11941/) (2022)
- [Smart Explorer: Recognizing Objects in Dense Clutter via Interactive Exploration](https://mlsanity.com/paper/2208.03496/) (2022)
- [Exact Solution Techniques for Two-dimensional Cutting and Packing](https://mlsanity.com/paper/2004.12619/) (2020)

## Built on by (most cited first, 8 of 8)

- [Prompt, Plan, Perform: LLM-based Humanoid Control via Quantized Imitation Learning](https://mlsanity.com/paper/2309.11359/) (2023)
- [Learning Physically Realizable Skills for Online Packing of General 3D Shapes](https://mlsanity.com/paper/2212.02094/) (2022)
- [Bin Packing Optimization via Deep Reinforcement Learning](https://mlsanity.com/paper/2403.12420/) (2024)
- [GFPack++: Improving 2D Irregular Packing by Learning Gradient Field with Attention](https://mlsanity.com/paper/2406.07579/) (2024)
- [Category-level Shape Estimation for Densely Cluttered Objects](https://mlsanity.com/paper/2302.11983/) (2023)
- [Learning Polynomial Representations of Physical Objects with Application to Certifying Correct Packing Configurations](https://mlsanity.com/paper/2312.06791/) (2023)
- [HERB: Human-augmented Efficient Reinforcement learning for Bin-packing](https://mlsanity.com/paper/2504.16595/) (2025)
- [OPA-Pack: Object-Property-Aware Robotic Bin Packing](https://mlsanity.com/paper/2505.13339/) (2025)

## Abstract (arXiv)

Object packing by autonomous robots is an im-portant challenge in warehouses and logistics industry. Most conventional data-driven packing planning approaches focus on regular cuboid packing, which are usually heuristic and limit the practical use in realistic applications with everyday objects. In this paper, we propose a deep hierarchical reinforcement learning approach to simultaneously plan packing sequence and placement for irregular object packing. Specifically, the top manager network infers packing sequence from six principal view heightmaps of all objects, and then the bottom worker network receives heightmaps of the next object to predict the placement position and orientation. The two networks are trained hierarchically in a self-supervised Q-Learning framework, where the rewards are provided by the packing results based on the top height , object volume and placement stability in the box. The framework repeats sequence and placement planning iteratively until all objects have been packed into the box or no space is remained for unpacked items. We compare our approach with existing robotic packing methods for irregular objects in a physics simulator. Experiments show that our approach can pack more objects with less time cost than the state-of-the-art packing methods of irregular objects. We also implement our packing plan with a robotic manipulator to show the generalization ability in the real world.