# TrajFusionNet: Pedestrian Crossing Intention Prediction via Fusion of Sequential and Visual Trajectory Representations

arXiv: https://arxiv.org/abs/2508.19866 · HTML: https://mlsanity.com/paper/2508.19866/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 48 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Visual Attention Network](https://mlsanity.com/paper/2202.09741/) (2022)
- [Deep Time Series Models: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2407.13278/) (2024)
- [Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction](https://mlsanity.com/paper/2002.08945/) (2020)
- [Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs](https://mlsanity.com/paper/2005.06582/) (2020)
- [Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention](https://mlsanity.com/paper/2104.05485/) (2021)
- [Real-time Intent Prediction of Pedestrians for Autonomous Ground Vehicles via Spatio-Temporal DenseNet](https://mlsanity.com/paper/1904.09862/) (2019)
- [Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction](https://mlsanity.com/paper/1910.09077/) (2019)
- [RNN-based Pedestrian Crossing Prediction using Activity and Pose-related Features](https://mlsanity.com/paper/2008.11647/) (2020)
- [Is attention to bounding boxes all you need for pedestrian action prediction?](https://mlsanity.com/paper/2107.08031/) (2021)
- [Coupling Intent and Action for Pedestrian Crossing Behavior Prediction](https://mlsanity.com/paper/2105.04133/) (2021)
- [TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction](https://mlsanity.com/paper/2109.00953/) (2021)
- [Action and intention recognition of pedestrians in urban traffic](https://mlsanity.com/paper/1810.09805/) (2018)
- [Deep Virtual-to-Real Distillation for Pedestrian Crossing Prediction](https://mlsanity.com/paper/2211.00856/) (2022)
- [VRUNet: Multi-Task Learning Model for Intent Prediction of Vulnerable Road Users](https://mlsanity.com/paper/2007.05397/) (2020)
- [Pedestrian Motion State Estimation From 2D Pose](https://mlsanity.com/paper/2103.00145/) (2021)

## Built on by (most cited first, 2 of 2)

- [KoopCast: Trajectory Forecasting via Koopman Operators](https://mlsanity.com/paper/2509.15513/) (2025)
- [TrajFusionNet+: Transformer-Based Prediction of Pedestrian Crossing Intention via Fusion of Trajectory Representations and Scene Graphs](https://mlsanity.com/paper/2609.10806/) (2026)

## Abstract (arXiv)

With the introduction of vehicles with autonomous capabilities on public roads, predicting pedestrian crossing intention has emerged as an active area of research. The task of predicting pedestrian crossing intention involves determining whether pedestrians in the scene are likely to cross the road or not. In this work, we propose TrajFusionNet, a novel transformer-based model that combines future pedestrian trajectory and vehicle speed predictions as priors for predicting crossing intention. TrajFusionNet comprises two branches: a Sequence Attention Module (SAM) and a Visual Attention Module (VAM). The SAM branch learns from a sequential representation of the observed and predicted pedestrian trajectory and vehicle speed. Complementarily, the VAM branch enables learning from a visual representation of the predicted pedestrian trajectory by overlaying predicted pedestrian bounding boxes onto scene images. By utilizing a small number of lightweight modalities, TrajFusionNet achieves the lowest total inference time (including model runtime and data preprocessing) among current state-of-the-art approaches. In terms of performance, it achieves state-of-the-art results across the three most commonly used datasets for pedestrian crossing intention prediction.