# Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly

arXiv: https://arxiv.org/abs/1910.13675 · HTML: https://mlsanity.com/paper/1910.13675/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2019.

## Built on by (most cited first, 30 of 51)

- [CLIPort: What and Where Pathways for Robotic Manipulation](https://mlsanity.com/paper/2109.12098/) (2021)
- [Visual Room Rearrangement](https://mlsanity.com/paper/2103.16544/) (2021)
- [Waypoint-Based Imitation Learning for Robotic Manipulation](https://mlsanity.com/paper/2307.14326/) (2023)
- [Learning to See before Learning to Act: Visual Pre-training for Manipulation](https://mlsanity.com/paper/2107.00646/) (2021)
- [O2O-Afford: Annotation-Free Large-Scale Object-Object Affordance Learning](https://mlsanity.com/paper/2106.15087/) (2021)
- [Equivariant Transporter Network](https://mlsanity.com/paper/2202.09400/) (2022)
- [Neural Shape Mating: Self-Supervised Object Assembly with Adversarial Shape Priors](https://mlsanity.com/paper/2205.14886/) (2022)
- [RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches](https://mlsanity.com/paper/2403.02709/) (2024)
- [JUICER: Data-Efficient Imitation Learning for Robotic Assembly](https://mlsanity.com/paper/2404.03729/) (2024)
- [RGL-NET: A Recurrent Graph Learning framework for Progressive Part Assembly](https://mlsanity.com/paper/2107.12859/) (2021)
- [Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations](https://mlsanity.com/paper/2408.04380/) (2024)
- [A Survey of Embodied Learning for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2408.11537/) (2024)
- [Leveraging SE(3) Equivariance for Learning 3D Geometric Shape Assembly](https://mlsanity.com/paper/2309.06810/) (2023)
- [Generalization Through Hand-Eye Coordination: An Action Space for Learning Spatially-Invariant Visuomotor Control](https://mlsanity.com/paper/2103.00375/) (2021)
- [Category-Level Multi-Part Multi-Joint 3D Shape Assembly](https://mlsanity.com/paper/2303.06163/) (2023)
- [Leveraging Symmetries in Pick and Place](https://mlsanity.com/paper/2308.07948/) (2023)
- [Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing](https://mlsanity.com/paper/2402.17464/) (2024)
- [Vision-driven Compliant Manipulation for Reliable, High-Precision Assembly Tasks](https://mlsanity.com/paper/2106.14070/) (2021)
- [ASAP: Automated Sequence Planning for Complex Robotic Assembly with Physical Feasibility](https://mlsanity.com/paper/2309.16909/) (2023)
- [Two by Two: Learning Multi-Task Pairwise Objects Assembly for Generalizable Robot Manipulation](https://mlsanity.com/paper/2504.06961/) (2025)
- [Automating Reinforcement Learning with Example-based Resets](https://mlsanity.com/paper/2204.02041/) (2022)
- [Rearrangement Planning for General Part Assembly](https://mlsanity.com/paper/2307.00206/) (2023)
- [SPAFormer: Sequential 3D Part Assembly with Transformers](https://mlsanity.com/paper/2403.05874/) (2024)
- [VLANeXt: Recipes for Building Strong VLA Models](https://mlsanity.com/paper/2602.18532/) (2026)
- [Action Priors for Large Action Spaces in Robotics](https://mlsanity.com/paper/2101.04178/) (2021)
- [IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes](https://mlsanity.com/paper/2112.05298/) (2021)
- [Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields](https://mlsanity.com/paper/2212.02493/) (2022)
- [3D Geometric Shape Assembly via Efficient Point Cloud Matching](https://mlsanity.com/paper/2407.10542/) (2024)
- [Multi-Task Learning with Sequence-Conditioned Transporter Networks](https://mlsanity.com/paper/2109.07578/) (2021)
- [Learning to Design and Construct Bridge without Blueprint](https://mlsanity.com/paper/2108.02439/) (2021)

## Abstract (arXiv)

Is it possible to learn policies for robotic assembly that can generalize to new objects? We explore this idea in the context of the kit assembly task. Since classic methods rely heavily on object pose estimation, they often struggle to generalize to new objects without 3D CAD models or task-specific training data. In this work, we propose to formulate the kit assembly task as a shape matching problem, where the goal is to learn a shape descriptor that establishes geometric correspondences between object surfaces and their target placement locations from visual input. This formulation enables the model to acquire a broader understanding of how shapes and surfaces fit together for assembly -- allowing it to generalize to new objects and kits. To obtain training data for our model, we present a self-supervised data-collection pipeline that obtains ground truth object-to-placement correspondences by disassembling complete kits. Our resulting real-world system, Form2Fit, learns effective pick and place strategies for assembling objects into a variety of kits -- achieving 90% average success rates under different initial conditions (e.g. varying object and kit poses), 94% success under new configurations of multiple kits, and over 86% success with completely new objects and kits.