# TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction

arXiv: https://arxiv.org/abs/2109.00953 · HTML: https://mlsanity.com/paper/2109.00953/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 55 references that are in the map.
- First posted on arXiv 2 Sep 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Lookahead Optimizer: k steps forward, 1 step back](https://mlsanity.com/paper/1907.08610/) (2019)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [Compressed Video Action Recognition](https://mlsanity.com/paper/1712.00636/) (2017)
- [Long-term Temporal Convolutions for Action Recognition](https://mlsanity.com/paper/1604.04494/) (2016)
- [Pose Flow: Efficient Online Pose Tracking](https://mlsanity.com/paper/1802.00977/) (2018)
- [Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty](https://mlsanity.com/paper/1711.09026/) (2017)
- [Real-time Action Recognition with Enhanced Motion Vector CNNs](https://mlsanity.com/paper/1604.07669/) (2016)
- [Pedestrian Action Anticipation using Contextual Feature Fusion in Stacked RNNs](https://mlsanity.com/paper/2005.06582/) (2020)
- [On the Integration of Optical Flow and Action Recognition](https://mlsanity.com/paper/1712.08416/) (2017)
- [Efficient Online Multi-Person 2D Pose Tracking with Recurrent Spatio-Temporal Affinity Fields](https://mlsanity.com/paper/1811.11975/) (2018)
- [Graph Edge Convolutional Neural Networks for Skeleton Based Action Recognition](https://mlsanity.com/paper/1805.06184/) (2018)
- [Real-time Intent Prediction of Pedestrians for Autonomous Ground Vehicles via Spatio-Temporal DenseNet](https://mlsanity.com/paper/1904.09862/) (2019)
- [Is the Pedestrian going to Cross? Answering by 2D Pose Estimation](https://mlsanity.com/paper/1807.10580/) (2018)
- [Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction](https://mlsanity.com/paper/1910.09077/) (2019)
- [Agreeing to Cross: How Drivers and Pedestrians Communicate](https://mlsanity.com/paper/1702.03555/) (2017)
- [Make Skeleton-based Action Recognition Model Smaller, Faster and Better](https://mlsanity.com/paper/1907.09658/) (2019)
- [FuSSI-Net: Fusion of Spatio-temporal Skeletons for Intention Prediction Network](https://mlsanity.com/paper/2005.07796/) (2020)

## Built on by (most cited first, 13 of 13)

- [Assessing Cross-dataset Generalization of Pedestrian Crossing Predictors](https://mlsanity.com/paper/2201.12626/) (2022)
- [PIP-Net: Pedestrian Intention Prediction in the Wild](https://mlsanity.com/paper/2402.12810/) (2024)
- [Deep Virtual-to-Real Distillation for Pedestrian Crossing Prediction](https://mlsanity.com/paper/2211.00856/) (2022)
- [Synthetic Data Generation Framework, Dataset, and Efficient Deep Model for Pedestrian Intention Prediction](https://mlsanity.com/paper/2401.06757/) (2024)
- [GTransPDM: A Graph-embedded Transformer with Positional Decoupling for Pedestrian Crossing Intention Prediction](https://mlsanity.com/paper/2409.20223/) (2024)
- [Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Survey](https://mlsanity.com/paper/2412.00348/) (2024)
- [Local and Global Contextual Features Fusion for Pedestrian Intention Prediction](https://mlsanity.com/paper/2305.01111/) (2023)
- [Multi-Context Fusion Transformer for Pedestrian Crossing Intention Prediction in Urban Environments](https://mlsanity.com/paper/2511.20011/) (2025)
- [ACIT: Attention-Guided Cross-Modal Interaction Transformer for Pedestrian Crossing Intention Prediction](https://mlsanity.com/paper/2511.20020/) (2025)
- [TrajFusionNet: Pedestrian Crossing Intention Prediction via Fusion of Sequential and Visual Trajectory Representations](https://mlsanity.com/paper/2508.19866/) (2025)
- [ESIA: An Energy-Based Spatiotemporal Interaction-Aware Framework for Pedestrian Intention Prediction](https://mlsanity.com/paper/2604.23728/) (2026)
- [Adaptive Cross-Modal Fusion with Sparse Attention for Pedestrian Crossing Intention Prediction](https://mlsanity.com/paper/2607.12293/) (2026)
- [TrajFusionNet+: Transformer-Based Prediction of Pedestrian Crossing Intention via Fusion of Trajectory Representations and Scene Graphs](https://mlsanity.com/paper/2609.10806/) (2026)

## Abstract (arXiv)

Understanding the behaviors and intentions of pedestrians is still one of the main challenges for vehicle autonomy, as accurate predictions of their intentions can guarantee their safety and driving comfort of vehicles. In this paper, we address pedestrian crossing prediction in urban traffic environments by linking the dynamics of a pedestrian's skeleton to a binary crossing intention. We introduce TrouSPI-Net: a context-free, lightweight, multi-branch predictor. TrouSPI-Net extracts spatio-temporal features for different time resolutions by encoding pseudo-images sequences of skeletal joints' positions and processes them with parallel attention modules and atrous convolutions. The proposed approach is then enhanced by processing features such as relative distances of skeletal joints, bounding box positions, or ego-vehicle speed with U-GRUs. Using the newly proposed evaluation procedures for two large public naturalistic data sets for studying pedestrian behavior in traffic: JAAD and PIE, we evaluate TrouSPI-Net and analyze its performance. Experimental results show that TrouSPI-Net achieved 0.76 F1 score on JAAD and 0.80 F1 score on PIE, therefore outperforming current state-of-the-art while being lightweight and context-free.