# Kit-Net: Self-Supervised Learning to Kit Novel 3D Objects into Novel 3D Cavities

arXiv: https://arxiv.org/abs/2107.05789 · HTML: https://mlsanity.com/paper/2107.05789/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 17 of 17)

- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Transporter Networks: Rearranging the Visual World for Robotic Manipulation](https://mlsanity.com/paper/2010.14406/) (2020)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking](https://mlsanity.com/paper/1905.09304/) (2019)
- [se(3)-TrackNet: Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains](https://mlsanity.com/paper/2007.13866/) (2020)
- [Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning](https://mlsanity.com/paper/1807.03146/) (2018)
- [6-DOF Grasping for Target-driven Object Manipulation in Clutter](https://mlsanity.com/paper/1912.03628/) (2019)
- [Generative 3D Part Assembly via Dynamic Graph Learning](https://mlsanity.com/paper/2006.07793/) (2020)
- [Form2Fit: Learning Shape Priors for Generalizable Assembly from Disassembly](https://mlsanity.com/paper/1910.13675/) (2019)
- [Relative Camera Pose Estimation Using Convolutional Neural Networks](https://mlsanity.com/paper/1702.01381/) (2017)
- [Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transforms](https://mlsanity.com/paper/2105.14246/) (2021)
- [Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning](https://mlsanity.com/paper/1709.06670/) (2017)
- [Learning Pose Estimation for High-Precision Robotic Assembly Using Simulated Depth Images](https://mlsanity.com/paper/1809.10699/) (2018)
- [Exploratory Grasping: Asymptotically Optimal Algorithms for Grasping Challenging Polyhedral Objects](https://mlsanity.com/paper/2011.05632/) (2020)
- [Interpreting Contact Interactions to Overcome Failure in Robot Assembly Tasks](https://mlsanity.com/paper/2101.02725/) (2021)
- [X-Ray: Mechanical Search for an Occluded Object by Minimizing Support of Learned Occupancy Distributions](https://mlsanity.com/paper/2004.09039/) (2020)

## Built on by (most cited first, 4 of 4)

- [Neural Shape Mating: Self-Supervised Object Assembly with Adversarial Shape Priors](https://mlsanity.com/paper/2205.14886/) (2022)
- [JUICER: Data-Efficient Imitation Learning for Robotic Assembly](https://mlsanity.com/paper/2404.03729/) (2024)
- [Leveraging SE(3) Equivariance for Learning 3D Geometric Shape Assembly](https://mlsanity.com/paper/2309.06810/) (2023)
- [Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assembly](https://mlsanity.com/paper/2110.04450/) (2021)

## Abstract (arXiv)

In industrial part kitting, 3D objects are inserted into cavities for transportation or subsequent assembly. Kitting is a critical step as it can decrease downstream processing and handling times and enable lower storage and shipping costs. We present Kit-Net, a framework for kitting previously unseen 3D objects into cavities given depth images of both the target cavity and an object held by a gripper in an unknown initial orientation. Kit-Net uses self-supervised deep learning and data augmentation to train a convolutional neural network (CNN) to robustly estimate 3D rotations between objects and matching concave or convex cavities using a large training dataset of simulated depth images pairs. Kit-Net then uses the trained CNN to implement a controller to orient and position novel objects for insertion into novel prismatic and conformal 3D cavities. Experiments in simulation suggest that Kit-Net can orient objects to have a 98.9% average intersection volume between the object mesh and that of the target cavity. Physical experiments with industrial objects succeed in 18% of trials using a baseline method and in 63% of trials with Kit-Net. Video, code, and data are available at https://github.com/BerkeleyAutomation/Kit-Net.