# Rearrangement Planning for General Part Assembly

arXiv: https://arxiv.org/abs/2307.00206 · HTML: https://mlsanity.com/paper/2307.00206/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 39 references that are in the map.
- First posted on arXiv 1 Jul 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Transporter Networks: Rearranging the Visual World for Robotic Manipulation](https://mlsanity.com/paper/2010.14406/) (2020)
- [Deep Closest Point: Learning Representations for Point Cloud Registration](https://mlsanity.com/paper/1905.03304/) (2019)
- [Geometric Transformer for Fast and Robust Point Cloud Registration](https://mlsanity.com/paper/2202.06688/) (2022)
- [Go-ICP: A Globally Optimal Solution to 3D ICP Point-Set Registration](https://mlsanity.com/paper/1605.03344/) (2016)
- [Rearrangement: A Challenge for Embodied AI](https://mlsanity.com/paper/2011.01975/) (2020)
- [Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols](https://mlsanity.com/paper/1502.03143/) (2015)
- [A comprehensive survey on point cloud registration](https://mlsanity.com/paper/2103.02690/) (2021)
- [Generative 3D Part Assembly via Dynamic Graph Learning](https://mlsanity.com/paper/2006.07793/) (2020)
- [A Large Dataset of Object Scans](https://mlsanity.com/paper/1602.02481/) (2016)
- [Factory: Fast Contact for Robotic Assembly](https://mlsanity.com/paper/2205.03532/) (2022)
- [Learning 3D Part Assembly from a Single Image](https://mlsanity.com/paper/2003.09754/) (2020)
- [Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly](https://mlsanity.com/paper/1910.13675/) (2019)
- [PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation](https://mlsanity.com/paper/1903.00709/) (2019)
- [Neural Shape Mating: Self-Supervised Object Assembly with Adversarial Shape Priors](https://mlsanity.com/paper/2205.14886/) (2022)
- [RGL-NET: A Recurrent Graph Learning framework for Progressive Part Assembly](https://mlsanity.com/paper/2107.12859/) (2021)
- [Learning to Group: A Bottom-Up Framework for 3D Part Discovery in Unseen Categories](https://mlsanity.com/paper/2002.06478/) (2020)
- [Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learning](https://mlsanity.com/paper/2203.13733/) (2022)
- [Learning Robotic Assembly from CAD](https://mlsanity.com/paper/1803.07635/) (2018)
- [RoboAssembly: Learning Generalizable Furniture Assembly Policy in a Novel Multi-robot Contact-rich Simulation Environment](https://mlsanity.com/paper/2112.10143/) (2021)
- [Robotic additive construction of bar structures: Unified sequence and motion planning](https://mlsanity.com/paper/2105.11438/) (2021)
- [AssembleRL: Learning to Assemble Furniture from Their Point Clouds](https://mlsanity.com/paper/2209.07268/) (2022)

## Built on by (most cited first, 12 of 12)

- [Rectified Point Flow: Generic Point Cloud Pose Estimation](https://mlsanity.com/paper/2506.05282/) (2025)
- [Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulation](https://mlsanity.com/paper/2504.06961/) (2025)
- [SPAFormer: Sequential 3D Part Assembly with Transformers](https://mlsanity.com/paper/2403.05874/) (2024)
- [Manual-PA: Learning 3D Part Assembly from Instruction Diagrams](https://mlsanity.com/paper/2411.18011/) (2024)
- [Multi-level Reasoning for Robotic Assembly: From Sequence Inference to Contact Selection](https://mlsanity.com/paper/2312.10571/) (2023)
- [SCANet: Correcting LEGO Assembly Errors with Self-Correct Assembly Network](https://mlsanity.com/paper/2403.18195/) (2024)
- [AssemblyBench: Physics-Aware Assembly of Complex Industrial Objects](https://mlsanity.com/paper/2605.12845/) (2026)
- [Component Selection for Craft Assembly Tasks](https://mlsanity.com/paper/2407.14001/) (2024)
- [Query-Centric Diffusion Policy for Generalizable Robotic Assembly](https://mlsanity.com/paper/2509.18686/) (2025)
- [Prompt2Craft: Generating Functional Craft Assemblies with LLMs](https://mlsanity.com/paper/2512.04568/) (2025)
- [PVRA: A Pointwise Key-point Voting Framework for Robotic Assembly](https://mlsanity.com/paper/2608.19968/) (2026)
- [PART: Learning 3D Part Assembly and Retrieval with Transformers](https://mlsanity.com/paper/2609.19872/) (2026)

## Abstract (arXiv)

Most successes in autonomous robotic assembly have been restricted to single target or category. We propose to investigate general part assembly, the task of creating novel target assemblies with unseen part shapes. As a fundamental step to a general part assembly system, we tackle the task of determining the precise poses of the parts in the target assembly, which we we term ``rearrangement planning''. We present General Part Assembly Transformer (GPAT), a transformer-based model architecture that accurately predicts part poses by inferring how each part shape corresponds to the target shape. Our experiments on both 3D CAD models and real-world scans demonstrate GPAT's generalization abilities to novel and diverse target and part shapes.