# IA-LSTM: Interaction-Aware LSTM for Pedestrian Trajectory Prediction

arXiv: https://arxiv.org/abs/2311.15193 · HTML: https://mlsanity.com/paper/2311.15193/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 114 references that are in the map.
- First posted on arXiv 26 Nov 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Deep Visual-Semantic Alignments for Generating Image Descriptions](https://mlsanity.com/paper/1412.2306/) (2014)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [Recent Trends in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/1708.02709/) (2017)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Prediction](https://mlsanity.com/paper/1903.02793/) (2019)
- [End-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First Results](https://mlsanity.com/paper/1412.1602/) (2014)
- [PORCA: Modeling and Planning for Autonomous Driving among Many Pedestrians](https://mlsanity.com/paper/1805.11833/) (2018)
- [Top-view Trajectories: A Pedestrian Dataset of Vehicle-Crowd Interaction from Controlled Experiments and Crowded Campus](https://mlsanity.com/paper/1902.00487/) (2019)
- [Crowded Scene Analysis: A Survey](https://mlsanity.com/paper/1502.01812/) (2015)
- [Pedestrian Trajectory Prediction in Pedestrian-Vehicle Mixed Environments: A Systematic Review](https://mlsanity.com/paper/2308.06419/) (2023)
- [MX-LSTM: mixing tracklets and vislets to jointly forecast trajectories and head poses](https://mlsanity.com/paper/1805.00652/) (2018)
- [Anticipating Visual Representations from Unlabeled Video](https://mlsanity.com/paper/1504.08023/) (2015)
- [STGlow: A Flow-based Generative Framework with Dual Graphormer for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2211.11220/) (2022)
- [Knowledge Transfer for Scene-specific Motion Prediction](https://mlsanity.com/paper/1603.06987/) (2016)
- [Tree Memory Networks for Modelling Long-term Temporal Dependencies](https://mlsanity.com/paper/1703.04706/) (2017)
- [Deep Learning Driven Visual Path Prediction from a Single Image](https://mlsanity.com/paper/1601.07265/) (2016)
- [Robust Learning with Kernel Mean p-Power Error Loss](https://mlsanity.com/paper/1612.07019/) (2016)
- [Multi-Kernel Correntropy for Robust Learning](https://mlsanity.com/paper/1905.10115/) (2019)
- [Early Recognition of Human Activities from First-Person Videos Using Onset Representations](https://mlsanity.com/paper/1406.5309/) (2014)

## Built on by (most cited first, 3 of 3)

- [Sparse Pedestrian Character Learning for Trajectory Prediction](https://mlsanity.com/paper/2311.15512/) (2023)
- [ART: Adaptive Relational Transformer for Pedestrian Trajectory Prediction with Temporal-Aware Relations](https://mlsanity.com/paper/2604.03649/) (2026)
- [Trustworthy Pedestrian Trajectory Prediction via Pattern-Aware Interaction Modeling](https://mlsanity.com/paper/2507.13397/) (2025)

## Abstract (arXiv)

Predicting the trajectory of pedestrians in crowd scenarios is indispensable in self-driving or autonomous mobile robot field because estimating the future locations of pedestrians around is beneficial for policy decision to avoid collision. It is a challenging issue because humans have different walking motions, and the interactions between humans and objects in the current environment, especially between humans themselves, are complex. Previous researchers focused on how to model human-human interactions but neglected the relative importance of interactions. To address this issue, a novel mechanism based on correntropy is introduced. The proposed mechanism not only can measure the relative importance of human-human interactions but also can build personal space for each pedestrian. An interaction module including this data-driven mechanism is further proposed. In the proposed module, the data-driven mechanism can effectively extract the feature representations of dynamic human-human interactions in the scene and calculate the corresponding weights to represent the importance of different interactions. To share such social messages among pedestrians, an interaction-aware architecture based on long short-term memory network for trajectory prediction is designed. Experiments are conducted on two public datasets. Experimental results demonstrate that our model can achieve better performance than several latest methods with good performance.