# Pose and Semantic Map Based Probabilistic Forecast of Vulnerable Road Users' Trajectories

arXiv: https://arxiv.org/abs/2106.02598 · HTML: https://mlsanity.com/paper/2106.02598/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 34 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Wider or Deeper: Revisiting the ResNet Model for Visual Recognition](https://mlsanity.com/paper/1611.10080/) (2016)
- [Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image](https://mlsanity.com/paper/1701.00295/) (2017)
- [MANTRA: Memory Augmented Networks for Multiple Trajectory Prediction](https://mlsanity.com/paper/2006.03340/) (2020)
- [MonoLayout: Amodal scene layout from a single image](https://mlsanity.com/paper/2002.08394/) (2020)
- [Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction](https://mlsanity.com/paper/1910.08041/) (2019)
- [Scene Compliant Trajectory Forecast with Agent-Centric Spatio-Temporal Grids](https://mlsanity.com/paper/1909.07507/) (2019)
- [Pedestrian Path, Pose and Intention Prediction through Gaussian Process Dynamical Models and Pedestrian Activity Recognition](https://mlsanity.com/paper/2004.14747/) (2020)
- [Intentions of Vulnerable Road Users - Detection and Forecasting by Means of Machine Learning](https://mlsanity.com/paper/1803.03577/) (2018)
- [MCENET: Multi-Context Encoder Network for Homogeneous Agent Trajectory Prediction in Mixed Traffic](https://mlsanity.com/paper/2002.05966/) (2020)
- [Quantile Surfaces -- Generalizing Quantile Regression to Multivariate Targets](https://mlsanity.com/paper/2010.05898/) (2020)

## Built on by (most cited first, 6 of 6)

- [Social-Transmotion: Promptable Human Trajectory Prediction](https://mlsanity.com/paper/2312.16168/) (2023)
- [Multi-Transmotion: Pre-trained Model for Human Motion Prediction](https://mlsanity.com/paper/2411.02673/) (2024)
- [Social-Pose: Enhancing Trajectory Prediction with Human Body Pose](https://mlsanity.com/paper/2507.22742/) (2025)
- [Feature Importance in Pedestrian Intention Prediction: A Context-Aware Review](https://mlsanity.com/paper/2409.07645/) (2024)
- [Likely, Light, and Accurate Context-Free Clusters-based Trajectory Prediction](https://mlsanity.com/paper/2307.14788/) (2023)
- [Three-Step Hierarchical Transformer for Multi-Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2606.23058/) (2026)

## Abstract (arXiv)

In this article, an approach for probabilistic trajectory forecasting of vulnerable road users (VRUs) is presented, which considers past movements and the surrounding scene. Past movements are represented by 3D poses reflecting the posture and movements of individual body parts. The surrounding scene is modeled in the form of semantic maps showing, e.g., the course of streets, sidewalks, and the occurrence of obstacles. The forecasts are generated in grids discretizing the space and in the form of arbitrary discrete probability distributions. The distributions are evaluated in terms of their reliability, sharpness, and positional accuracy. We compare our method with an approach that provides forecasts in the form of Gaussian distributions and discuss the respective advantages and disadvantages. Thereby, we investigate the impact of using poses and semantic maps. With a technique called spatial label smoothing, our approach achieves reliable forecasts. Overall, the poses have a positive impact on the forecasts. The semantic maps offer the opportunity to adapt the probability distributions to the individual situation, although at the considered forecasted time horizon of 2.52 s they play a minor role compared to the past movements of the VRU. Our method is evaluated on a dataset recorded in inner-city traffic using a research vehicle. The dataset is made publicly available.