# SRA-LSTM: Social Relationship Attention LSTM for Human Trajectory Prediction

arXiv: https://arxiv.org/abs/2103.17045 · HTML: https://mlsanity.com/paper/2103.17045/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 44 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints](https://mlsanity.com/paper/1806.01482/) (2018)
- [Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks](https://mlsanity.com/paper/1907.03395/) (2019)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [Peeking into the Future: Predicting Future Person Activities and Locations in Videos](https://mlsanity.com/paper/1902.03748/) (2019)
- [SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Prediction](https://mlsanity.com/paper/1903.02793/) (2019)
- [Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANs](https://mlsanity.com/paper/1904.09507/) (2019)
- [Conditional Generative Neural System for Probabilistic Trajectory Prediction](https://mlsanity.com/paper/1905.01631/) (2019)
- [Recursive Social Behavior Graph for Trajectory Prediction](https://mlsanity.com/paper/2004.10402/) (2020)
- [Collaborative Motion Prediction via Neural Motion Message Passing](https://mlsanity.com/paper/2003.06594/) (2020)
- [Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event Detection](https://mlsanity.com/paper/1702.05552/) (2017)
- [Social and Scene-Aware Trajectory Prediction in Crowded Spaces](https://mlsanity.com/paper/1909.08840/) (2019)
- [An Attention-Based Deep Learning Model for Multiple Pedestrian Attributes Recognition](https://mlsanity.com/paper/2004.01110/) (2020)

## Built on by (most cited first, 3 of 3)

- [Pishgu: Universal Path Prediction Network Architecture for Real-time Cyber-physical Edge Systems](https://mlsanity.com/paper/2210.08057/) (2022)
- [CARLA-BSP: a simulated dataset with pedestrians](https://mlsanity.com/paper/2305.00204/) (2023)
- [CPSOR-GCN: A Vehicle Trajectory Prediction Method Powered by Emotion and Cognitive Theory](https://mlsanity.com/paper/2311.08086/) (2023)

## Abstract (arXiv)

Pedestrian trajectory prediction for surveillance video is one of the important research topics in the field of computer vision and a key technology of intelligent surveillance systems. Social relationship among pedestrians is a key factor influencing pedestrian walking patterns but was mostly ignored in the literature. Pedestrians with different social relationships play different roles in the motion decision of target pedestrian. Motivated by this idea, we propose a Social Relationship Attention LSTM (SRA-LSTM) model to predict future trajectories. We design a social relationship encoder to obtain the representation of their social relationship through the relative position between each pair of pedestrians. Afterwards, the social relationship feature and latent movements are adopted to acquire the social relationship attention of this pair of pedestrians. Social interaction modeling is achieved by utilizing social relationship attention to aggregate movement information from neighbor pedestrians. Experimental results on two public walking pedestrian video datasets (ETH and UCY), our model achieves superior performance compared with state-of-the-art methods. Contrast experiments with other attention methods also demonstrate the effectiveness of social relationship attention.