CLAP: Cross-Embodiment Video World Models are Zero-Shot Physical Simulators
On the map: World models → Embodied and robot world models
Abstract (arXiv)
State-of-the-art action-conditioned video models are typically restricted to a single robot embodiment, preventing them from leveraging the vast corpus of heterogeneous video data that contains rich signals for learning generalizable physics. To bridge this gap, we introduce CLAP, a framework for cross-embodiment action-conditioned video generation capable of being trained on diverse, internet-scale videos across human and robotic agents. CLAP is grounded in the insight that universal physical laws govern spatiotemporal dynamics regardless of the actor. However, cross-embodiment learning is non-trivial because action representations vary sharply across robot platforms and are typically absent in human videos. CLAP addresses this fundamental challenge through the following core contributions. First, CLAP reconciles disparate action spaces using end-effector poses, language instructions, and latent actions. Second, to resolve their individual limitations, CLAP introduces a curriculum-based cross-embodiment learning recipe that first learns foundational physical priors across unlabeled video data using latent actions and subsequently grounds them in end-effector action spaces for zero-shot deployment to real-world tasks. Crucially, CLAP approaches or surpasses state-of-the-art single-embodiment video models in challenging environments like DROID. These performance advantages compound via few-shot adaptation to establish a novel paradigm for training single-embodiment video world models. Ultimately, CLAP delivers the most comprehensive suite of action-conditioned video world models to date - spanning diverse action-conditioning spaces (end-effector, language, and latent) and robot morphologies (including cross-embodiment, DROID, Bridge, bimanual YAM robots, and G1 humanoids). We open-source all code and models. Project Website at https://omni-clap.github.io .
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 53 of its 67 references that are in the map. First posted on arXiv 27 Aug 2026.
Builds on
53 of 67 references.
Closest ancestors · 13
Same direction, newest first
- 2026Hydra-0: Action Flow for Generalist World Modeling and Control18 Aug
- 2026Masked Visual Actions for Unified World Modeling21 Jul
- 2026WEAVER, Better, Faster, Longer: An Effective World Model for Robotic Manipulation11 Jun
- 2026PlayWorld: Learning Robot World Models from Autonomous Play9 Mar
- 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos6 Feb
- 2025Evaluating Gemini Robotics Policies in a Veo World Simulator11 Dec
- 2025World Models That Know When They Don't Know - Controllable Video Generation with Calibrated Uncertainty5 Dec
- 2025Scalable Policy Evaluation with Video World Models14 Nov
- 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulation11 Oct
- 2025villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models31 Jul
- 2025WorldGym: World Model as An Environment for Policy Evaluation31 May
- 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Models19 May
1 more
Other references · 40
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
28 more
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
- 2024Genie: Generative Interactive Environmentscited by 706
- 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
- 2024Latent Action Pretraining from Videoscited by 302
- 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
- 2025Unified Video Action Modelcited by 195
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 191
- 2025World Simulation with Video Foundation Models for Physical AIcited by 183
- 2023Video Language Planningcited by 160
- 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
- 2024Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformerscited by 145
- 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
- 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
- 2024Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviationcited by 108
- 2025Universal Actions for Enhanced Embodied Foundation Modelscited by 80
- 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgettingcited by 52
- 2026RoboReward: General-Purpose Vision-Language Reward Models for Roboticscited by 46
- 2026MolmoAct2: Action Reasoning Models for Real-world Deploymentcited by 45
- 2025CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrationscited by 39
- 2026LAP: Language-Action Pre-Training Enables Zero-shot Cross-Embodiment Transfercited by 17
- 2026Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directionscited by 12
- 2025How Confident are Video Models? Empowering Video Models to Express their Uncertaintycited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026XEWorld: Can Action-Conditioned World Models Generalize to Unseen Robot Embodiments?
- 2026LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models
- 2026Dream4ACT: A Shared Visual Action Interface for Multi-Embodiment Video-Action Modeling
- 2026Latent Policy Steering: An Efficient and Flexible Framework for Cross-Embodiment Transfer ↗
- 2026World Action Models are Zero-shot Policies
- 2026Bridging the Embodiment Gap: Disentangled Cross-Embodiment Video Editing ↗
- 2025TraceGen: World Modeling in 3D Trace Space Enables Learning from Cross-Embodiment Videos
- 2026BridgeV2W: Bridging Video Generation Models to Embodied World Models via Embodiment Masks