Towards Human Motion World Models via Executable Behaviour Representations
On the map: World models → Embodied and robot world models
Abstract (arXiv)
Human motion world models should capture motion's intentionality by being executable: adaptable to different actions and capable of assessing motion quality. To achieve this, we introduce a domain-specific language ExAct that represents human motions as underspecified programs that can be compiled to a reward model for zero-shot policy inference. By leveraging the compositional nature of ExAct programs, we combine individual policies into executable behaviour representations. We evaluate the utility of the proposed approach by analysing human motion capture for the tasks of human action segmentation and human action anomaly detection. Our results suggest that the improvement in data efficiency and the capture of intuitive relationships between human actions are better than those of task-specific models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 24 of its 38 references that are in the map. First posted on arXiv 20 Apr 2026.
Builds on
24 of 38 references.
References · 24
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023Visual Instruction Tuningcited by 7,792
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
- 2024Qwen2.5-Coder Technical Reportcited by 1,151
- 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
- 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
- 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
- 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
- 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
12 more
- 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 198
- 2023Universal Humanoid Motion Representations for Physics-Based Controlcited by 109
- 2021Learning Successor States and Goal-Dependent Values: A Mathematical Viewpointcited by 67
- 2024SynCode: LLM Generation with Grammar Augmentationcited by 66
- 2022Normalizing Flows for Human Pose Anomaly Detectioncited by 59
- 2023Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligencecited by 52
- 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
- 2021Iterative Contrast-Classify For Semi-supervised Temporal Action Segmentationcited by 19
- 2021Hierarchical Motion Understanding via Motion Programscited by 14
- 2024Goals as Reward-Producing Programscited by 14
- 2022Programmatic Concept Learning for Human Motion Description and Synthesiscited by 5
- 2025EPFL-Smart-Kitchen-30: Densely annotated cooking dataset with 3D kinematics to challenge video and language modelscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026XPACE: Joint World and Action Modeling from Heterogeneous Experience
- 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
- 2026UniWAM: Unified World-Action Model
- 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videos
- 2026WatchAct: A Benchmark for Behavior-Grounded Robot Manipulation
- 2026WorldReward: Reward Modeling for Camera-Conditioned World Models
- 2026WLA3: World Latent Action Modeling for Semantics, Dynamics, and Kinematics
- 2026From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data