# Learning Sim-Grounded Policies for Bimanual Rope Manipulation from Human Teleoperation Data

arXiv: https://arxiv.org/abs/2605.16043 · HTML: https://mlsanity.com/paper/2605.16043/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 15 May 2026.

## Builds on (most cited first, 12 of 12)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors](https://mlsanity.com/paper/2210.11339/) (2022)
- [Learning Generalizable Manipulation Policies with Object-Centric 3D Representations](https://mlsanity.com/paper/2310.14386/) (2023)
- [Generalizable Humanoid Manipulation with 3D Diffusion Policies](https://mlsanity.com/paper/2410.10803/) (2024)
- [Challenges and Outlook in Robotic Manipulation of Deformable Objects](https://mlsanity.com/paper/2105.01767/) (2021)
- [Untangling Dense Knots by Learning Task-Relevant Keypoints](https://mlsanity.com/paper/2011.04999/) (2020)
- [Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policies](https://mlsanity.com/paper/2107.08942/) (2021)
- [Autonomously Untangling Long Cables](https://mlsanity.com/paper/2207.07813/) (2022)
- [GaussTwin: Unified Simulation and Correction with Gaussian Splatting for Robotic Digital Twins](https://mlsanity.com/paper/2603.05108/) (2026)
- [PIPHEN: Physical Interaction Prediction with Hamiltonian Energy Networks](https://mlsanity.com/paper/2511.16200/) (2025)

## Abstract (arXiv)

Deformable Linear Objects (DLOs) such as ropes and cables are widely encountered in both household and industrial applications, yet remain challenging to manipulate due to their infinite-dimensional configuration space and frequent self-occlusion. Imitation learning from teleoperation offers a practical path to bimanual DLO manipulation, but its scalability is limited by human effort, making the choice of observation space critical for generalization from small datasets. In this study, we investigate whether the lack of generalization in egocentric visual policies for the knot-untangling task stems from the observation space itself, rather than from the policy architecture or data scale. We compare two Action Chunking with Transformers policies trained on the same bimanual teleoperation data: a vision-based policy conditioned on two egocentric RGB streams from wrist-mounted cameras, and a state-based policy conditioned on the DLO's 3D particle state, extracted from an initial observation via multi-view fusion and evolved in a particle-based eXtended Position-Based Dynamics simulation. Evaluated open-loop on an unseen rope configuration, the state-based policy outperforms its visual counterpart with a 30.8% reduction in L1 error when predicting the initial grasp-and-pull action, quantifying the observability gap between pixels and physics-consistent state, and pointing toward more data-efficient robot learning for the DLO manipulation task from limited human demonstrations.