# RegHEC: Hand-Eye Calibration via Simultaneous Multi-view Point Clouds Registration of Arbitrary Object

arXiv: https://arxiv.org/abs/2304.14092 · HTML: https://mlsanity.com/paper/2304.14092/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 85 references that are in the map.
- First posted on arXiv 27 Apr 2023.

## Builds on (most cited first, 9 of 9)

- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Robots that can adapt like animals](https://mlsanity.com/paper/1407.3501/) (2014)
- [A micro Lie theory for state estimation in robotics](https://mlsanity.com/paper/1812.01537/) (2018)
- [AA-ICP: Iterative Closest Point with Anderson Acceleration](https://mlsanity.com/paper/1709.05479/) (2017)
- [Solving the Robot-World Hand-Eye(s) Calibration Problem with Iterative Methods](https://mlsanity.com/paper/1907.12425/) (2019)
- [Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics](https://mlsanity.com/paper/1709.06917/) (2017)
- [Anderson Acceleration for Geometry Optimization and Physics Simulation](https://mlsanity.com/paper/1805.05715/) (2018)
- [Self-Calibration of Mobile Manipulator Kinematic and Sensor Extrinsic Parameters Through Contact-Based Interaction](https://mlsanity.com/paper/1803.06406/) (2018)
- [In Situ Translational Hand-Eye Calibration of Laser Profile Sensors using Arbitrary Objects](https://mlsanity.com/paper/2103.11981/) (2021)

## Built on by (most cited first, 1 of 1)

- [Hydra: Marker-Free RGB-D Hand-Eye Calibration](https://mlsanity.com/paper/2504.20584/) (2025)

## Abstract (arXiv)

RegHEC is a registration-based hand-eye calibration technique with no need for accurate calibration rig but arbitrary available objects, applicable for both eye-in-hand and eye-to-hand cases. It tries to find the hand-eye relation which brings multi-view point clouds of arbitrary scene into simultaneous registration under a common reference frame. RegHEC first achieves initial alignment of multi-view point clouds via Bayesian optimization, where registration problem is modeled as a Gaussian process over hand-eye relation and the covariance function is modified to be compatible with distance metric in 3-D motion space SE(3), then passes the initial guess of hand-eye relation to an Anderson Accelerated ICP variant for later fine registration and accurate calibration. RegHEC has little requirement on calibration object, it is applicable with sphere, cone, cylinder and even simple plane, which can be quite challenging for correct point cloud registration and sensor motion estimation using existing methods. While suitable for most 3-D vision guided tasks, RegHEC is especially favorable for robotic 3-D reconstruction, as calibration and multi-view point clouds registration of reconstruction target are unified into a single process. Our technique is verified with extensive experiments using varieties of arbitrary objects and real hand-eye system. We release an open-source C++ implementation of RegHEC.