# CFVS: Coarse-to-Fine Visual Servoing for 6-DoF Object-Agnostic Peg-In-Hole Assembly

arXiv: https://arxiv.org/abs/2209.08864 · HTML: https://mlsanity.com/paper/2209.08864/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Sep 2022.

## Built on by (most cited first, 6 of 6)

- [ForceSight: Text-Guided Mobile Manipulation with Visual-Force Goals](https://mlsanity.com/paper/2309.12312/) (2023)
- [Evaluating Robustness of Visual Representations for Object Assembly Task Requiring Spatio-Geometrical Reasoning](https://mlsanity.com/paper/2310.09943/) (2023)
- [DeLTa: Demonstration and Language-Guided Novel Transparent Object Manipulation](https://mlsanity.com/paper/2510.05662/) (2025)
- [Reliability-aware Execution Gating for Near-field and Off-axis Vision-guided Robotic Alignment](https://mlsanity.com/paper/2602.08466/) (2026)
- [Context-Based Meta Reinforcement Learning for Robust and Adaptable Peg-in-Hole Assembly Tasks](https://mlsanity.com/paper/2409.16208/) (2024)
- [3D Reconstruction Techniques in the Manufacturing Domain: Applications, Research Opportunities and Use Cases](https://mlsanity.com/paper/2604.28064/) (2026)

## Abstract (arXiv)

Robotic peg-in-hole assembly remains a challenging task due to its high accuracy demand. Previous work tends to simplify the problem by restricting the degree of freedom of the end-effector, or limiting the distance between the target and the initial pose position, which prevents them from being deployed in real-world manufacturing. Thus, we present a Coarse-to-Fine Visual Servoing (CFVS) peg-in-hole method, achieving 6-DoF end-effector motion control based on 3D visual feedback. CFVS can handle arbitrary tilt angles and large initial alignment errors through a fast pose estimation before refinement. Furthermore, by introducing a confidence map to ignore the irrelevant contour of objects, CFVS is robust against noise and can deal with various targets beyond training data. Extensive experiments show CFVS outperforms state-of-the-art methods and obtains 100%, 91%, and 82% average success rates in 3-DoF, 4-DoF, and 6-DoF peg-in-hole, respectively.