# Forecasting People Trajectories and Head Poses by Jointly Reasoning on Tracklets and Vislets

arXiv: https://arxiv.org/abs/1901.02000 · HTML: https://mlsanity.com/paper/1901.02000/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2019.

## Built on by (most cited first, 13 of 13)

- [Synthesis of Compositional Animations from Textual Descriptions](https://mlsanity.com/paper/2103.14675/) (2021)
- [Social-Transmotion: Promptable Human Trajectory Prediction](https://mlsanity.com/paper/2312.16168/) (2023)
- [Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-based Approaches](https://mlsanity.com/paper/2111.06740/) (2021)
- [Lifting Monocular Events to 3D Human Poses](https://mlsanity.com/paper/2104.10609/) (2021)
- [Self-Supervised 3D Human Pose Estimation with Multiple-View Geometry](https://mlsanity.com/paper/2108.07777/) (2021)
- [D2-TPred: Discontinuous Dependency for Trajectory Prediction under Traffic Lights](https://mlsanity.com/paper/2207.10398/) (2022)
- [Under the Hood of Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2203.11878/) (2022)
- [Social Processes: Self-Supervised Meta-Learning over Conversational Groups for Forecasting Nonverbal Social Cues](https://mlsanity.com/paper/2107.13576/) (2021)
- [Social-Pose: Enhancing Trajectory Prediction with Human Body Pose](https://mlsanity.com/paper/2507.22742/) (2025)
- [Learning Temporal 3D Human Pose Estimation with Pseudo-Labels](https://mlsanity.com/paper/2110.07578/) (2021)
- [Towards more realistic human motion prediction with attention to motion coordination](https://mlsanity.com/paper/2404.03584/) (2024)
- [Visual Domain Adaptation for Monocular Depth Estimation on Resource-Constrained Hardware](https://mlsanity.com/paper/2108.02671/) (2021)
- [Attention-based Vehicle Self-Localization with HD Feature Maps](https://mlsanity.com/paper/2107.07787/) (2021)

## Abstract (arXiv)

In this work, we explore the correlation between people trajectories and their head orientations. We argue that people trajectory and head pose forecasting can be modelled as a joint problem. Recent approaches on trajectory forecasting leverage short-term trajectories (aka tracklets) of pedestrians to predict their future paths. In addition, sociological cues, such as expected destination or pedestrian interaction, are often combined with tracklets. In this paper, we propose MiXing-LSTM (MX-LSTM) to capture the interplay between positions and head orientations (vislets) thanks to a joint unconstrained optimization of full covariance matrices during the LSTM backpropagation. We additionally exploit the head orientations as a proxy for the visual attention, when modeling social interactions. MX-LSTM predicts future pedestrians location and head pose, increasing the standard capabilities of the current approaches on long-term trajectory forecasting. Compared to the state-of-the-art, our approach shows better performances on an extensive set of public benchmarks. MX-LSTM is particularly effective when people move slowly, i.e. the most challenging scenario for all other models. The proposed approach also allows for accurate predictions on a longer time horizon.