# Diving Deeper Into Pedestrian Behavior Understanding: Intention Estimation, Action Prediction, and Event Risk Assessment

arXiv: https://arxiv.org/abs/2407.00446 · HTML: https://mlsanity.com/paper/2407.00446/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 29 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [Autonomous Vehicles that Interact with Pedestrians: A Survey of Theory and Practice](https://mlsanity.com/paper/1805.11773/) (2018)
- [Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction](https://mlsanity.com/paper/2002.08945/) (2020)
- [TITAN: Future Forecast using Action Priors](https://mlsanity.com/paper/2003.13886/) (2020)
- [GANet: Goal Area Network for Motion Forecasting](https://mlsanity.com/paper/2209.09723/) (2022)
- [Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs](https://mlsanity.com/paper/2005.06582/) (2020)
- [Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention](https://mlsanity.com/paper/2104.05485/) (2021)
- [Bifold and Semantic Reasoning for Pedestrian Behavior Prediction](https://mlsanity.com/paper/2012.03298/) (2020)
- [Real-time Intent Prediction of Pedestrians for Autonomous Ground Vehicles via Spatio-Temporal DenseNet](https://mlsanity.com/paper/1904.09862/) (2019)
- [Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction](https://mlsanity.com/paper/1910.09077/) (2019)
- [Multi-Modal Hybrid Architecture for Pedestrian Action Prediction](https://mlsanity.com/paper/2012.00514/) (2020)
- [PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learning](https://mlsanity.com/paper/2210.07886/) (2022)
- [Is attention to bounding boxes all you need for pedestrian action prediction?](https://mlsanity.com/paper/2107.08031/) (2021)
- [CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationships](https://mlsanity.com/paper/2207.03586/) (2022)
- [PSI: A Benchmark for Human Interpretation and Response in Traffic Interactions](https://mlsanity.com/paper/2112.02604/) (2021)
- [Coupling Intent and Action for Pedestrian Crossing Behavior Prediction](https://mlsanity.com/paper/2105.04133/) (2021)
- [Graph-SIM: A Graph-based Spatiotemporal Interaction Modelling for Pedestrian Action Prediction](https://mlsanity.com/paper/2012.02148/) (2020)
- [A Novel Benchmarking Paradigm and a Scale- and Motion-Aware Model for Egocentric Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2310.10424/) (2023)
- [Pedestrian Behavior Prediction for Automated Driving: Requirements, Metrics, and Relevant Features](https://mlsanity.com/paper/2012.08418/) (2020)
- [Assessing Cross-dataset Generalization of Pedestrian Crossing Predictors](https://mlsanity.com/paper/2201.12626/) (2022)
- [PePScenes: A Novel Dataset and Baseline for Pedestrian Action Prediction in 3D](https://mlsanity.com/paper/2012.07773/) (2020)
- [Cross or Wait? Predicting Pedestrian Interaction Outcomes at Unsignalized Crossings](https://mlsanity.com/paper/2304.08260/) (2023)
- [Pedestrian Stop and Go Forecasting with Hybrid Feature Fusion](https://mlsanity.com/paper/2203.02489/) (2022)
- [DESTINE: Dynamic Goal Queries with Temporal Transductive Alignment for Trajectory Prediction](https://mlsanity.com/paper/2310.07438/) (2023)
- [Learning Pedestrian Actions to Ensure Safe Autonomous Driving](https://mlsanity.com/paper/2305.13051/) (2023)

## Built on by (most cited first, 2 of 2)

- [SEPose: A Synthetic Event-based Human Pose Estimation Dataset for Pedestrian Monitoring](https://mlsanity.com/paper/2507.11910/) (2025)
- [Unified Vision-Centric Pedestrian Crossing Action Prediction via Adaptive Patch Projection and Proactive Spatial Rectification](https://mlsanity.com/paper/2609.07420/) (2026)

## Abstract (arXiv)

In this paper, we delve into the pedestrian behavior understanding problem from the perspective of three different tasks: intention estimation, action prediction, and event risk assessment. We first define the tasks and discuss how these tasks are represented and annotated in two widely used pedestrian datasets, JAAD and PIE. We then propose a new benchmark based on these definitions, available annotations, and three new classes of metrics, each designed to assess different aspects of the model performance. We apply the new evaluation approach to examine four SOTA prediction models on each task and compare their performance w.r.t. metrics and input modalities. In particular, we analyze the differences between intention estimation and action prediction tasks by considering various scenarios and contextual factors. Lastly, we examine model agreement across these two tasks to show their complementary role. The proposed benchmark reveals new facts about the role of different data modalities, the tasks, and relevant data properties. We conclude by elaborating on our findings and proposing future research directions.