# Human Pose Forecasting via Deep Markov Models

arXiv: https://arxiv.org/abs/1707.09240 · HTML: https://mlsanity.com/paper/1707.09240/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2017.

## Built on by (most cited first, 16 of 16)

- [HA-ViD: A Human Assembly Video Dataset for Comprehensive Assembly Knowledge Understanding](https://mlsanity.com/paper/2307.05721/) (2023)
- [ATTACH Dataset: Annotated Two-Handed Assembly Actions for Human Action Understanding](https://mlsanity.com/paper/2304.08210/) (2023)
- [IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videos](https://mlsanity.com/paper/2411.11409/) (2024)
- [Aligning Step-by-Step Instructional Diagrams to Video Demonstrations](https://mlsanity.com/paper/2303.13800/) (2023)
- [Human Action Anticipation: A Survey](https://mlsanity.com/paper/2410.14045/) (2024)
- [IKEA Object State Dataset: A 6DoF object pose estimation dataset and benchmark for multi-state assembly objects](https://mlsanity.com/paper/2111.08614/) (2021)
- [Are current long-term video understanding datasets long-term?](https://mlsanity.com/paper/2308.11244/) (2023)
- [On the Stochastic Stability of Deep Markov Models](https://mlsanity.com/paper/2111.04601/) (2021)
- [Overview of Tencent Multi-modal Ads Video Understanding Challenge](https://mlsanity.com/paper/2109.07951/) (2021)
- [Tencent AVS: A Holistic Ads Video Dataset for Multi-modal Scene Segmentation](https://mlsanity.com/paper/2212.04700/) (2022)
- [Data Collection-free Masked Video Modeling](https://mlsanity.com/paper/2409.06665/) (2024)
- [GoferBot: A Visual Guided Human-Robot Collaborative Assembly System](https://mlsanity.com/paper/2304.08840/) (2023)
- [Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding in Novel Domains](https://mlsanity.com/paper/2311.18773/) (2023)
- [3D Pose Nowcasting: Forecast the Future to Improve the Present](https://mlsanity.com/paper/2308.12914/) (2023)
- [Dual-Path Temporal Map Optimization for Make-up Temporal Video Grounding](https://mlsanity.com/paper/2309.06176/) (2023)
- [Personalized Pose Forecasting](https://mlsanity.com/paper/2312.03528/) (2023)

## Abstract (arXiv)

Human pose forecasting is an important problem in computer vision with applications to human-robot interaction, visual surveillance, and autonomous driving. Usually, forecasting algorithms use 3D skeleton sequences and are trained to forecast for a few milliseconds into the future. Long-range forecasting is challenging due to the difficulty of estimating how long a person continues an activity. To this end, our contributions are threefold: (i) we propose a generative framework for poses using variational autoencoders based on Deep Markov Models (DMMs); (ii) we evaluate our pose forecasts using a pose-based action classifier, which we argue better reflects the subjective quality of pose forecasts than distance in coordinate space; (iii) last, for evaluation of the new model, we introduce a 480,000-frame video dataset called Ikea Furniture Assembly (Ikea FA), which depicts humans repeatedly assembling and disassembling furniture. We demonstrate promising results for our approach on both Ikea FA and the existing NTU RGB+D dataset.