# AssemblyComplete: 3D Combinatorial Construction with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2410.15469 · HTML: https://mlsanity.com/paper/2410.15469/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 20 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 15 of 15)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generation](https://mlsanity.com/paper/2206.12055/) (2022)
- [Multimodal Shape Completion via Conditional Generative Adversarial Networks](https://mlsanity.com/paper/2003.07717/) (2020)
- [Brick-by-Brick: Combinatorial Construction with Deep Reinforcement Learning](https://mlsanity.com/paper/2110.15481/) (2021)
- [3D Part Assembly Generation with Instance Encoded Transformer](https://mlsanity.com/paper/2207.01779/) (2022)
- [Combinatorial 3D Shape Generation via Sequential Assembly](https://mlsanity.com/paper/2004.07414/) (2020)
- [Planning Assembly Sequence with Graph Transformer](https://mlsanity.com/paper/2210.05236/) (2022)
- [StableLego: Stability Analysis of Block Stacking Assembly](https://mlsanity.com/paper/2402.10711/) (2024)
- [Physics-Aware Combinatorial Assembly Sequence Planning using Data-free Action Masking](https://mlsanity.com/paper/2408.10162/) (2024)
- [Simulation-aided Learning from Demonstration for Robotic LEGO Construction](https://mlsanity.com/paper/2309.11010/) (2023)
- [SERoCS: Safe and Efficient Robot Collaborative Systems for Next Generation Intelligent Industrial Co-Robots](https://mlsanity.com/paper/1809.08215/) (2018)
- [Robust and Context-Aware Real-Time Collaborative Robot Handling via Dynamic Gesture Commands](https://mlsanity.com/paper/2304.06175/) (2023)

## Built on by (most cited first, 1 of 1)

- [Rollback-Free Stable Brick Structures Generation](https://mlsanity.com/paper/2605.06947/) (2026)

## Abstract (arXiv)

A critical goal in robotics and autonomy is to teach robots to adapt to real-world collaborative tasks, particularly in automatic assembly. The ability of a robot to understand the original intent of an incomplete assembly and complete missing features without human instruction is valuable but challenging. This paper introduces 3D combinatorial assembly completion, which is demonstrated using combinatorial unit primitives (i.e., Lego bricks). Combinatorial assembly is challenging due to the possible assembly combinations and complex physical constraints (e.g., no brick collisions, structure stability, inventory constraints, etc.). To address these challenges, we propose a two-part deep reinforcement learning (DRL) framework that tackles teaching the robot to understand the objective of an incomplete assembly and learning a construction policy to complete the assembly. The robot queries a stable object library to facilitate assembly inference and guide learning. In addition to the robot policy, an action mask is developed to rule out invalid actions that violate physical constraints for object-oriented construction. We demonstrate the proposed framework's feasibility and robustness in a variety of assembly scenarios in which the robot satisfies real-life assembly with respect to both solution and runtime quality. Furthermore, results demonstrate that the proposed framework effectively infers and assembles incomplete structures for unseen and unique object types.