# Calib3R: Hand-Eye Calibration and 3D Metric-Scaled Scene Reconstruction with 3D Foundation Models

arXiv: https://arxiv.org/abs/2509.08813 · HTML: https://mlsanity.com/paper/2509.08813/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2025.

## Built on by (most cited first, 1 of 1)

- [Multi-Camera Self-Calibration in Sports Motion Capture: Leveraging Human and Stick Poses](https://mlsanity.com/paper/2604.17567/) (2026)

## Abstract (arXiv)

Robots often rely on RGB images for tasks like manipulation. However, reliable interaction typically requires a 3D scene representation that is metric-scaled and aligned with the robot reference frame. This depends on accurate hand-eye calibration and dense 3D reconstruction, tasks usually treated separately, despite both relying on geometric correspondences from RGB data. Traditional calibration techniques needs patterns, while RGB-based reconstruction yields 3D geometry with an unknown scale in an arbitrary frame. Multi-camera setups add further complexity, as data must be expressed in a shared reference frame. We present Calib3R, a patternless method that jointly performs hand-eye calibration and metric-scaled 3D reconstruction via unified optimization. Calib3R handles single- and multi-camera setups on robot arms. It builds on a 3D foundation model to extract pointmaps from RGB images, which are combined with robot poses to reconstruct a scaled 3D scene aligned with the robot base reference frame. Experiments on diverse datasets show that Calib3R achieves accurate calibration with less than 10 images, outperforming patternless and pattern-based methods.