# Emotion-Conditioned Short-Horizon Human Pose Forecasting with a Lightweight Predictive World Model

arXiv: https://arxiv.org/abs/2604.23532 · HTML: https://mlsanity.com/paper/2604.23532/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 5 of its 11 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 5 of 5)

- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [BlazePose: On-device Real-time Body Pose tracking](https://mlsanity.com/paper/2006.10204/) (2020)
- [Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs](https://mlsanity.com/paper/1907.06724/) (2019)
- [Action-Agnostic Human Pose Forecasting](https://mlsanity.com/paper/1810.09676/) (2018)
- [Hierarchical World Models as Visual Whole-Body Humanoid Controllers](https://mlsanity.com/paper/2405.18418/) (2024)

## Built on by (most cited first, 1 of 1)

- [Learning When to Listen: Gated Affect Fusion for Human Motion Prediction](https://mlsanity.com/paper/2607.00296/) (2026)

## Abstract (arXiv)

Short-term human pose prediction plays a crucial role in interactive systems, assistive robots, and emotion-aware human-computer interaction[1-3]. While current trajectory prediction models primarily rely on geometric motion cues, they often overlook the underlying emotional signals influencing human motion dynamics[4-5]. This paper investigates whether facial expression-derived emotion embeddings can provide auxiliary conditional signals for short-term pose prediction. To further evaluate multimodal conditionation in a recursive prediction setting, we propose a lightweight autoregressive predictive world model that performs 15-step rolling pose prediction. This framework combines pose keypoints with emotion embeddings through a learnable gating mechanism and performs autoregressive unfolding prediction using a recurrent sequence model based on a two-layer LSTM architecture. Experiments were conducted on two small-scale pose-emotion video datasets: controlled motion sequences with minimal facial expression changes and, natural emotion-driven motion sequences with considerable facial expression changes. The results show that simple multimodal fusion does not consistently improve prediction accuracy, while normalized gating fusion significantly enhances the performance of emotion-driven motion sequences. Furthermore, counterfactual perturbation experiments demonstrate that the predicted trajectory exhibits measurable sensitivity to changes in multimodal input, suggesting that facial expression embeddings act as auxiliary conditional signals rather than redundant features. In summary, these results indicate that incorporating facial expression-derived emotion embeddings into emotion-conditional short-term pose prediction based on a lightweight predictive world model architecture is a feasible approach.