# Learning Trajectory-Conditioned Relations to Predict Pedestrian Crossing Behavior

arXiv: https://arxiv.org/abs/2301.05796 · HTML: https://mlsanity.com/paper/2301.05796/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 18 references that are in the map.
- First posted on arXiv 14 Jan 2023.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs](https://mlsanity.com/paper/2005.06582/) (2020)
- [Joint Attention in Autonomous Driving (JAAD)](https://mlsanity.com/paper/1609.04741/) (2016)
- [Real-time Intent Prediction of Pedestrians for Autonomous Ground Vehicles via Spatio-Temporal DenseNet](https://mlsanity.com/paper/1904.09862/) (2019)
- [Multi-Modal Hybrid Architecture for Pedestrian Action Prediction](https://mlsanity.com/paper/2012.00514/) (2020)
- [Novelty Detection Through Model-Based Characterization of Neural Networks](https://mlsanity.com/paper/2008.06094/) (2020)
- [Pedestrian Intention Prediction: A Multi-task Perspective](https://mlsanity.com/paper/2010.10270/) (2020)
- [Coupling Intent and Action for Pedestrian Crossing Behavior Prediction](https://mlsanity.com/paper/2105.04133/) (2021)
- [Extracting Causal Visual Features for Limited label Classification](https://mlsanity.com/paper/2103.12322/) (2021)
- [Relative Afferent Pupillary Defect Screening through Transfer Learning](https://mlsanity.com/paper/1908.02300/) (2019)
- [Open-Set Recognition with Gradient-Based Representations](https://mlsanity.com/paper/2206.08229/) (2022)
- [Man-recon: manifold learning for reconstruction with deep autoencoder for smart seismic interpretation](https://mlsanity.com/paper/2212.07568/) (2022)
- [Explaining Deep Models through Forgettable Learning Dynamics](https://mlsanity.com/paper/2301.04221/) (2023)
- [On the Structures of Representation for the Robustness of Semantic Segmentation to Input Corruption](https://mlsanity.com/paper/2009.00817/) (2020)

## Abstract (arXiv)

In smart transportation, intelligent systems avoid potential collisions by predicting the intent of traffic agents, especially pedestrians. Pedestrian intent, defined as future action, e.g., start crossing, can be dependent on traffic surroundings. In this paper, we develop a framework to incorporate such dependency given observed pedestrian trajectory and scene frames. Our framework first encodes regional joint information between a pedestrian and surroundings over time into feature-map vectors. The global relation representations are then extracted from pairwise feature-map vectors to estimate intent with past trajectory condition. We evaluate our approach on two public datasets and compare against two state-of-the-art approaches. The experimental results demonstrate that our method helps to inform potential risks during crossing events with 0.04 improvement in F1-score on JAAD dataset and 0.01 improvement in recall on PIE dataset. Furthermore, we conduct ablation experiments to confirm the contribution of the relation extraction in our framework.