# In-Hand Pose Estimation and Pin Inspection for Insertion of Through-Hole Components

arXiv: https://arxiv.org/abs/2208.01284 · HTML: https://mlsanity.com/paper/2208.01284/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 2 Aug 2022.

## Builds on (most cited first, 8 of 8)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [BOP Challenge 2020 on 6D Object Localization](https://mlsanity.com/paper/2009.07378/) (2020)
- [Robust, Occlusion-aware Pose Estimation for Objects Grasped by Adaptive Hands](https://mlsanity.com/paper/2003.03518/) (2020)
- [Siamese Convolutional Neural Network for Sub-millimeter-accurate Camera Pose Estimation and Visual Servoing](https://mlsanity.com/paper/1903.04713/) (2019)
- [In-Hand Object Pose Tracking via Contact Feedback and GPU-Accelerated Robotic Simulation](https://mlsanity.com/paper/2002.12160/) (2020)
- [Deep learning classification of large-scale point clouds: A case study on cuneiform tablets](https://mlsanity.com/paper/2202.10851/) (2022)
- [ParaPose: Parameter and Domain Randomization Optimization for Pose Estimation using Synthetic Data](https://mlsanity.com/paper/2203.00945/) (2022)

## Built on by (most cited first, 3 of 3)

- [KeyMatchNet: Zero-Shot Pose Estimation in 3D Point Clouds by Generalized Keypoint Matching](https://mlsanity.com/paper/2303.16102/) (2023)
- [Good Grasps Only: A data engine for self-supervised fine-tuning of pose estimation using grasp poses for verification](https://mlsanity.com/paper/2409.11512/) (2024)
- [Quality Inspection of Printed Circuit Board Pin Insertion via Semantic Segmentation and Board-Level Feature Extraction](https://mlsanity.com/paper/2608.22937/) (2026)

## Abstract (arXiv)

The insertion of through-hole components is a difficult task. As the tolerances of the holes are very small, minor errors in the insertion will result in failures. These failures can damage components and will require manual intervention for recovery. Errors can occur both from imprecise object grasps and bent pins. Therefore, it is important that a system can accurately determine the object's position and reject components with bent pins. By utilizing the constraints inherent in the object grasp a method using template matching is able to obtain very precise pose estimates. Methods for pin-checking are also implemented, compared, and a successful method is shown. The set-up is performed automatically, with two novel contributions. A deep learning segmentation of the pins is performed and the inspection pose is found by simulation. From the inspection pose and the segmented pins, the templates for pose estimation and pin check are then generated. To train the deep learning method a dataset of segmented through-hole components is created. The network shows a 97.3 % accuracy on the test set. The pin-segmentation network is also tested on the insertion CAD models and successfully segment the pins. The complete system is tested on three different objects, and experiments show that the system is able to insert all objects successfully. Both by correcting in-hand grasp errors and rejecting objects with bent pins.