# Deep Sensorimotor Control by Imitating Predictive Models of Human Motion

arXiv: https://arxiv.org/abs/2508.18691 · HTML: https://mlsanity.com/paper/2508.18691/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2025.

## Built on by (most cited first, 4 of 4)

- [Interactive World Simulator for Robot Policy Training and Evaluation](https://mlsanity.com/paper/2603.08546/) (2026)
- [Joint-Aligned Latent Action: Towards Scalable VLA Pretraining in the Wild](https://mlsanity.com/paper/2602.21736/) (2026)
- [Robot Learning from Human Videos: A Survey](https://mlsanity.com/paper/2604.27621/) (2026)
- [EgoExo-WM: Unlocking Exo Video for Ego World Models](https://mlsanity.com/paper/2605.15477/) (2026)

## Abstract (arXiv)

As the embodiment gap between a robot and a human narrows, new opportunities arise to leverage datasets of humans interacting with their surroundings for robot learning. We propose a novel technique for training sensorimotor policies with reinforcement learning by imitating predictive models of human motions. Our key insight is that the motion of keypoints on human-inspired robot end-effectors closely mirrors the motion of corresponding human body keypoints. This enables us to use a model trained to predict future motion on human data zero-shot on robot data. We train sensorimotor policies to track the predictions of such a model, conditioned on a history of past robot states, while optimizing a relatively sparse task reward. This approach entirely bypasses gradient-based kinematic retargeting and adversarial losses, which limit existing methods from fully leveraging the scale and diversity of modern human-scene interaction datasets. Empirically, we find that our approach can work across robots and tasks, outperforming existing baselines by a large margin. In addition, we find that tracking a human motion model can substitute for carefully designed dense rewards and curricula in manipulation tasks. Code, data and qualitative results available at https://jirl-upenn.github.io/trackreward/.