# RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation

arXiv: https://arxiv.org/abs/2607.06558 · HTML: https://mlsanity.com/paper/2607.06558/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 34 of its 53 references that are in the map.
- First posted on arXiv 7 Jul 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 30 of 34)

- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [Pyramidal Flow Matching for Efficient Video Generative Modeling](https://mlsanity.com/paper/2410.05954/) (2024)
- [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026)
- [EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video](https://mlsanity.com/paper/2505.11709/) (2025)
- [World Simulation with Video Foundation Models for Physical AI](https://mlsanity.com/paper/2511.00062/) (2025)
- [TWIST: Teleoperated Whole-Body Imitation System](https://mlsanity.com/paper/2505.02833/) (2025)
- [Phantom: Training Robots Without Robots Using Only Human Videos](https://mlsanity.com/paper/2503.00779/) (2025)
- [Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videos](https://mlsanity.com/paper/2510.21571/) (2025)
- [Masquerade: Learning from In-the-wild Human Videos using Data-Editing](https://mlsanity.com/paper/2508.09976/) (2025)
- [World Models for Learning Dexterous Hand-Object Interactions from Human Videos](https://mlsanity.com/paper/2512.13644/) (2025)
- [H2R: A Human-to-Robot Data Augmentation for Robot Pre-training from Videos](https://mlsanity.com/paper/2505.11920/) (2025)
- [PlayerOne: Egocentric World Simulator](https://mlsanity.com/paper/2506.09995/) (2025)
- [Hand2World: Autoregressive Egocentric Interaction Generation via Free-Space Hand Gestures](https://mlsanity.com/paper/2602.09600/) (2026)
- [Mitty: Diffusion-based Human-to-Robot Video Generation](https://mlsanity.com/paper/2512.17253/) (2025)
- [EgoSim: Egocentric World Simulator for Embodied Interaction Generation](https://mlsanity.com/paper/2604.01001/) (2026)
- [HFGS: 4D Gaussian Splatting with Emphasis on Spatial and Temporal High-Frequency Components for Endoscopic Scene Reconstruction](https://mlsanity.com/paper/2405.17872/) (2024)
- [Generated Reality: Human-centric World Simulation using Interactive Video Generation with Hand and Camera Control](https://mlsanity.com/paper/2602.18422/) (2026)
- [LOME: Learning Human-Object Manipulation with Action-Conditioned Egocentric World Model](https://mlsanity.com/paper/2603.27449/) (2026)
- [X-Humanoid: Robotize Human Videos to Generate Humanoid Videos at Scale](https://mlsanity.com/paper/2512.04537/) (2025)

## Built on by (most cited first, 3 of 3)

- [EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPE](https://mlsanity.com/paper/2607.28243/) (2026)
- [KineWorld: Action-Induced Transport Fields for Embodied World Modeling](https://mlsanity.com/paper/2610.06349/) (2026)
- [SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models](https://mlsanity.com/paper/2610.06598/) (2026)

## Abstract (arXiv)

Scaling robot learning requires massive, diverse trajectory data, yet collection is currently bottlenecked by physical teleoperation, where every demonstration binds operator time to specific hardware and workspaces. We introduce digital teleoperation, a paradigm that decouples data collection from physical constraints by replacing the real robot with a generative world model. In this framework, an operator's hand-pose stream drives a robot-centric generative world model to synthesize high-fidelity egocentric videos from a single reference image. The recorded pose stream serves as an embodiment-agnostic action label transferable to any target robot via standard retargeting, yielding complete state-action trajectories for imitation learning independent of physical hardware. We instantiate this paradigm in RynnWorld-Teleop, a system that integrates depth-aware skeletal conditioning, progressive human-to-robot training on a video Diffusion Transformer (DiT), and streaming autoregressive distillation. This pipeline compresses the generative process into a single-pass inference, enabling 40+ FPS, real-time interactive generation on a single H100 GPU. Policies trained exclusively on RynnWorld-Teleop-generated data achieve effective zero-shot Sim2Real transfer across dexterous and diverse bimanual tasks. Moreover, augmenting real-world datasets with our digitally teleoperated data consistently improves success rates, demonstrating that RynnWorld-Teleop serves as a high-fidelity, scalable data engine for the next generation of robotic agents.