# RGL-NET: A Recurrent Graph Learning framework for Progressive Part Assembly

arXiv: https://arxiv.org/abs/2107.12859 · HTML: https://mlsanity.com/paper/2107.12859/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 27 Jul 2021.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions](https://mlsanity.com/paper/1905.06292/) (2019)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [SDM-NET: Deep Generative Network for Structured Deformable Mesh](https://mlsanity.com/paper/1908.04520/) (2019)
- [BAE-NET: Branched Autoencoder for Shape Co-Segmentation](https://mlsanity.com/paper/1903.11228/) (2019)
- [Generative 3D Part Assembly via Dynamic Graph Learning](https://mlsanity.com/paper/2006.07793/) (2020)
- [StructureNet: Hierarchical Graph Networks for 3D Shape Generation](https://mlsanity.com/paper/1908.00575/) (2019)
- [Learning 3D Part Assembly from a Single Image](https://mlsanity.com/paper/2003.09754/) (2020)
- [Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly](https://mlsanity.com/paper/1910.13675/) (2019)
- [Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assembly](https://mlsanity.com/paper/1903.01066/) (2019)
- [Learning Part Generation and Assembly for Structure-aware Shape Synthesis](https://mlsanity.com/paper/1906.06693/) (2019)
- [Composite Shape Modeling via Latent Space Factorization](https://mlsanity.com/paper/1901.02968/) (2019)
- [COALESCE: Component Assembly by Learning to Synthesize Connections](https://mlsanity.com/paper/2008.01936/) (2020)
- [GRASS: Generative Recursive Autoencoders for Shape Structures](https://mlsanity.com/paper/1705.02090/) (2017)
- [TearingNet: Point Cloud Autoencoder to Learn Topology-Friendly Representations](https://mlsanity.com/paper/2006.10187/) (2020)
- [SAGNet:Structure-aware Generative Network for 3D-Shape Modeling](https://mlsanity.com/paper/1808.03981/) (2018)
- [PT2PC: Learning to Generate 3D Point Cloud Shapes from Part Tree Conditions](https://mlsanity.com/paper/2003.08624/) (2020)
- [ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis](https://mlsanity.com/paper/2009.08026/) (2020)
- [ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling](https://mlsanity.com/paper/1708.01841/) (2017)
- [Learning to Scaffold the Development of Robotic Manipulation Skills](https://mlsanity.com/paper/1911.00969/) (2019)

## Built on by (most cited first, 30 of 36)

- [JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints](https://mlsanity.com/paper/2111.12772/) (2021)
- [SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects](https://mlsanity.com/paper/2410.16499/) (2024)
- [Neural Shape Mating: Self-Supervised Object Assembly with Adversarial Shape Priors](https://mlsanity.com/paper/2205.14886/) (2022)
- [Breaking Bad: A Dataset for Geometric Fracture and Reassembly](https://mlsanity.com/paper/2210.11463/) (2022)
- [Jigsaw: Learning to Assemble Multiple Fractured Objects](https://mlsanity.com/paper/2305.17975/) (2023)
- [Leveraging SE(3) Equivariance for Learning 3D Geometric Shape Assembly](https://mlsanity.com/paper/2309.06810/) (2023)
- [IKEA-Manual: Seeing Shape Assembly Step by Step](https://mlsanity.com/paper/2302.01881/) (2023)
- [Category-Level Multi-Part Multi-Joint 3D Shape Assembly](https://mlsanity.com/paper/2303.06163/) (2023)
- [3D Part Assembly Generation with Instance Encoded Transformer](https://mlsanity.com/paper/2207.01779/) (2022)
- [Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing](https://mlsanity.com/paper/2402.17464/) (2024)
- [Re-assembling the past: The RePAIR dataset and benchmark for real world 2D and 3D puzzle solving](https://mlsanity.com/paper/2410.24010/) (2024)
- [Fantastic Breaks: A Dataset of Paired 3D Scans of Real-World Broken Objects and Their Complete Counterparts](https://mlsanity.com/paper/2303.14152/) (2023)
- [PuzzleFusion++: Auto-agglomerative 3D Fracture Assembly by Denoise and Verify](https://mlsanity.com/paper/2406.00259/) (2024)
- [Scalable Geometric Fracture Assembly via Co-creation Space among Assemblers](https://mlsanity.com/paper/2312.12340/) (2023)
- [Geometric Deep Learning for Computer-Aided Design: A Survey](https://mlsanity.com/paper/2402.17695/) (2024)
- [Rectified Point Flow: Generic Point Cloud Pose Estimation](https://mlsanity.com/paper/2506.05282/) (2025)
- [Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulation](https://mlsanity.com/paper/2504.06961/) (2025)
- [Rearrangement Planning for General Part Assembly](https://mlsanity.com/paper/2307.00206/) (2023)
- [Score-PA: Score-based 3D Part Assembly](https://mlsanity.com/paper/2309.04220/) (2023)
- [SPAFormer: Sequential 3D Part Assembly with Transformers](https://mlsanity.com/paper/2403.05874/) (2024)
- [IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos](https://mlsanity.com/paper/2411.11409/) (2024)
- [Assembler: Scalable 3D Part Assembly via Anchor Point Diffusion](https://mlsanity.com/paper/2506.17074/) (2025)
- [ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents](https://mlsanity.com/paper/2510.21432/) (2025)
- [3D Geometric Shape Assembly via Efficient Point Cloud Matching](https://mlsanity.com/paper/2407.10542/) (2024)
- [A Survey on Computational Solutions for Reconstructing Complete Objects by Reassembling Their Fractured Parts](https://mlsanity.com/paper/2410.14770/) (2024)
- [Manual-PA: Learning 3D Part Assembly from Instruction Diagrams](https://mlsanity.com/paper/2411.18011/) (2024)
- [Jigsaw++: Imagining Complete Shape Priors for Object Reassembly](https://mlsanity.com/paper/2410.11816/) (2024)
- [Geometric Point Attention Transformer for 3D Shape Reassembly](https://mlsanity.com/paper/2411.17788/) (2024)
- [Interact3D: Compositional 3D Generation of Interactive Objects](https://mlsanity.com/paper/2603.16085/) (2026)
- [AssembleRL: Learning to Assemble Furniture from Their Point Clouds](https://mlsanity.com/paper/2209.07268/) (2022)

## Abstract (arXiv)

Autonomous assembly of objects is an essential task in robotics and 3D computer vision. It has been studied extensively in robotics as a problem of motion planning, actuator control and obstacle avoidance. However, the task of developing a generalized framework for assembly robust to structural variants remains relatively unexplored. In this work, we tackle this problem using a recurrent graph learning framework considering inter-part relations and the progressive update of the part pose. Our network can learn more plausible predictions of shape structure by accounting for priorly assembled parts. Compared to the current state-of-the-art, our network yields up to 10% improvement in part accuracy and up to 15% improvement in connectivity accuracy on the PartNet dataset. Moreover, our resulting latent space facilitates exciting applications such as shape recovery from the point-cloud components. We conduct extensive experiments to justify our design choices and demonstrate the effectiveness of the proposed framework.