# Fast robust peg-in-hole insertion with continuous visual servoing

arXiv: https://arxiv.org/abs/2011.06399 · HTML: https://mlsanity.com/paper/2011.06399/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [Residual Feedback Learning for Contact-Rich Manipulation Tasks with Uncertainty](https://mlsanity.com/paper/2106.04306/) (2021)
- [Coarse-to-Fine for Sim-to-Real: Sub-Millimetre Precision Across Wide Task Spaces](https://mlsanity.com/paper/2105.11283/) (2021)
- [Learning Eye-in-Hand Camera Calibration from a Single Image](https://mlsanity.com/paper/2111.01245/) (2021)
- [PolyFit: A Peg-in-hole Assembly Framework for Unseen Polygon Shapes via Sim-to-real Adaptation](https://mlsanity.com/paper/2312.02531/) (2023)
- [Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holes](https://mlsanity.com/paper/2205.04297/) (2022)
- [Cognitive Manipulation: Semi-supervised Visual Representation and Classroom-to-real Reinforcement Learning for Assembly in Semi-structured Environments](https://mlsanity.com/paper/2406.00364/) (2024)
- [Context-Based Meta Reinforcement Learning for Robust and Adaptable Peg-in-Hole Assembly Tasks](https://mlsanity.com/paper/2409.16208/) (2024)
- [Zero-Shot Peg Insertion: Identifying Mating Holes and Estimating SE(2) Poses with Vision-Language Models](https://mlsanity.com/paper/2503.06026/) (2025)
- [SI-Diff: A Framework for Learning Search and High-Precision Insertion with a Force-Domain Diffusion Policy](https://mlsanity.com/paper/2605.12247/) (2026)

## Abstract (arXiv)

This paper demonstrates a visual servoing method which is robust towards uncertainties related to system calibration and grasping, while significantly reducing the peg-in-hole time compared to classical methods and recent attempts based on deep learning. The proposed visual servoing method is based on peg and hole point estimates from a deep neural network in a multi-cam setup, where the model is trained on purely synthetic data. Empirical results show that the learnt model generalizes to the real world, allowing for higher success rates and lower cycle times than existing approaches.