# WTTFNet: A Weather-Time-Trajectory Fusion Network for Pedestrian Trajectory Prediction in Urban Complex

arXiv: https://arxiv.org/abs/2405.18945 · HTML: https://mlsanity.com/paper/2405.18945/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 54 references that are in the map.
- First posted on arXiv 29 May 2024.

## Builds on (most cited first, 12 of 12)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints](https://mlsanity.com/paper/1806.01482/) (2018)
- [Gated Multimodal Units for Information Fusion](https://mlsanity.com/paper/1702.01992/) (2017)
- [Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction](https://mlsanity.com/paper/2002.08945/) (2020)
- [BiTraP: Bi-directional Pedestrian Trajectory Prediction with Multi-modal Goal Estimation](https://mlsanity.com/paper/2007.14558/) (2020)
- [Deep Supervision with Intermediate Concepts](https://mlsanity.com/paper/1801.03399/) (2018)
- [FuSSI-Net: Fusion of Spatio-temporal Skeletons for Intention Prediction Network](https://mlsanity.com/paper/2005.07796/) (2020)
- [Social Force Embedded Mixed Graph Convolutional Network for Multi-class Trajectory Prediction](https://mlsanity.com/paper/2404.13378/) (2024)
- [How to restart? An agent-based simulation model towards the definition of strategies for COVID-19 "second phase" in public buildings](https://mlsanity.com/paper/2004.12927/) (2020)

## Abstract (arXiv)

Pedestrian trajectory modelling in an urban complex is challenging because pedestrians can have many possible destinations, such as shops, escalators, and attractions. Moreover, weather and time-of-day may affect pedestrian behavior. In this paper, a new weather-time-trajectory fusion network (WTTFNet) is proposed to improve the performance of baseline deep neural network architecture. By incorporating weather and time-of-day information as an embedding structure, a novel WTTFNet based on gate multimodal unit is used to fuse the multimodal information and deep representation of trajectories. A joint loss function based on focal loss is used to co-optimize both the deep trajectory features and final classifier, which helps to improve the accuracy in predicting the intended destination of pedestrians and hence the trajectories under possible scenarios of class imbalances. Experimental results using the Osaka Asia and Pacific Trade Center (ATC) dataset shows improved performance of the proposed approach over state-of-the-art algorithms by 23.67% increase in classification accuracy, 9.16% and 7.07% reduction of average and final displacement error. The proposed approach may serve as an attractive approach for improving existing baseline trajectory prediction models when they are applied to scenarios with influences of weather-time conditions. It can be employed in numerous applications such as pedestrian facility engineering, public space development and technology-driven retail.