# Pedestrian Trajectory Prediction with Structured Memory Hierarchies

arXiv: https://arxiv.org/abs/1807.08381 · HTML: https://mlsanity.com/paper/1807.08381/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2018.

## Built on by (most cited first, 2 of 2)

- [Using Auxiliary Information for Person Re-Identification -- A Tutorial Overview](https://mlsanity.com/paper/2211.08565/) (2022)
- [TSCA-Net: Temporal-Spatial Clique Attention for Interpretable Multimodal Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2607.11939/) (2026)

## Abstract (arXiv)

This paper presents a novel framework for human trajectory prediction based on multimodal data (video and radar). Motivated by recent neuroscience discoveries, we propose incorporating a structured memory component in the human trajectory prediction pipeline to capture historical information to improve performance. We introduce structured LSTM cells for modelling the memory content hierarchically, preserving the spatiotemporal structure of the information and enabling us to capture both short-term and long-term context. We demonstrate how this architecture can be extended to integrate salient information from multiple modalities to automatically store and retrieve important information for decision making without any supervision. We evaluate the effectiveness of the proposed models on a novel multimodal dataset that we introduce, consisting of 40,000 pedestrian trajectories, acquired jointly from a radar system and a CCTV camera system installed in a public place. The performance is also evaluated on the publicly available New York Grand Central pedestrian database. In both settings, the proposed models demonstrate their capability to better anticipate future pedestrian motion compared to existing state of the art.