# Exploring Pose-Guided Imitation Learning for Robotic Precise Insertion

arXiv: https://arxiv.org/abs/2505.09424 · HTML: https://mlsanity.com/paper/2505.09424/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 0 of its 34 references that are in the map.
- First posted on arXiv 14 May 2025.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Built on by (most cited first, 2 of 2)

- [Investigating a Policy-Based Formulation for Endoscopic Camera Pose Recovery](https://mlsanity.com/paper/2603.20045/) (2026)
- [Dissecting Motion-Prior Regularization for Data-Scarce Robotic Insertion](https://mlsanity.com/paper/2609.17484/) (2026)

## Abstract (arXiv)

Imitation learning is promising for robotic manipulation, but precise insertion in the real world remains difficult due to contact-rich dynamics, tight clearances, and limited demonstrations. Many existing visuomotor policies depend on high-dimensional RGB/point-cloud observations, which can be data-inefficient and generalize poorly under pose variations. In this paper, we study pose-guided imitation learning by using object poses in SE(3) as compact, object-centric observations for precise insertion tasks. First, we propose a diffusion policy for precise insertion that observes the relative SE(3) pose of the source object with respect to the target object and predicts a future relative pose trajectory as its action. Second, to improve robustness to pose estimation noise, we augment the pose-guided policy with RGBD cues. Specifically, we introduce a goal-conditioned RGBD encoder to capture the discrepancy between current and goal observations. We further propose a pose-guided residual gated fusion module, where pose features provide the primary control signal and RGBD features adaptively compensate when pose estimates are unreliable. We evaluate our methods on six real-robot precise insertion tasks and achieve high performance with only 7--10 demonstrations per task. In our setup, the proposed policies succeed on tasks with clearances down to 0.01~mm and demonstrate improved data efficiency and generalization over existing baselines. Code will be available at https://github.com/sunhan1997/PoseInsert.