# Neural Packing: from Visual Sensing to Reinforcement Learning

arXiv: https://arxiv.org/abs/2311.09233 · HTML: https://mlsanity.com/paper/2311.09233/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 30 references that are in the map.
- First posted on arXiv 17 Oct 2023.

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Online 3D Bin Packing with Constrained Deep Reinforcement Learning](https://mlsanity.com/paper/2006.14978/) (2020)
- [Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method](https://mlsanity.com/paper/1708.05930/) (2017)
- [Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimization](https://mlsanity.com/paper/1807.01672/) (2018)
- [Stable bin packing of non-convex 3D objects with a robot manipulator](https://mlsanity.com/paper/1812.04093/) (2018)
- [TAP-Net: Transport-and-Pack using Reinforcement Learning](https://mlsanity.com/paper/2009.01469/) (2020)
- [Mononizing Binocular Videos](https://mlsanity.com/paper/2009.01424/) (2020)
- [Towards Zero-Waste Furniture Design](https://mlsanity.com/paper/1604.00047/) (2016)
- [Automated pebble mosaic stylization of images](https://mlsanity.com/paper/1902.02806/) (2019)

## Built on by (most cited first, 3 of 3)

- [GOPT: Generalizable Online 3D Bin Packing via Transformer-based Deep Reinforcement Learning](https://mlsanity.com/paper/2409.05344/) (2024)
- [Deliberate Planning of 3D Bin Packing on Packing Configuration Trees](https://mlsanity.com/paper/2504.04421/) (2025)
- [GFPack++: Improving 2D Irregular Packing by Learning Gradient Field with Attention](https://mlsanity.com/paper/2406.07579/) (2024)

## Abstract (arXiv)

We present a novel learning framework to solve the transport-and-packing (TAP) problem in 3D. It constitutes a full solution pipeline from partial observations of input objects via RGBD sensing and recognition to final box placement, via robotic motion planning, to arrive at a compact packing in a target container. The technical core of our method is a neural network for TAP, trained via reinforcement learning (RL), to solve the NP-hard combinatorial optimization problem. Our network simultaneously selects an object to pack and determines the final packing location, based on a judicious encoding of the continuously evolving states of partially observed source objects and available spaces in the target container, using separate encoders both enabled with attention mechanisms. The encoded feature vectors are employed to compute the matching scores and feasibility masks of different pairings of box selection and available space configuration for packing strategy optimization. Extensive experiments, including ablation studies and physical packing execution by a real robot (Universal Robot UR5e), are conducted to evaluate our method in terms of its design choices, scalability, generalizability, and comparisons to baselines, including the most recent RL-based TAP solution. We also contribute the first benchmark for TAP which covers a variety of input settings and difficulty levels.