# PMM-Net: Single-stage Multi-agent Trajectory Prediction with Patching-based Embedding and Explicit Modal Modulation

arXiv: https://arxiv.org/abs/2410.19544 · HTML: https://mlsanity.com/paper/2410.19544/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [A Time Series is Worth 64 Words: Long-term Forecasting with Transformers](https://mlsanity.com/paper/2211.14730/) (2022)
- [Are Transformers Effective for Time Series Forecasting?](https://mlsanity.com/paper/2205.13504/) (2022)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction](https://mlsanity.com/paper/2004.02025/) (2020)
- [Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion](https://mlsanity.com/paper/2203.13777/) (2022)
- [From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting](https://mlsanity.com/paper/2012.01526/) (2020)
- [Leapfrog Diffusion Model for Stochastic Trajectory Prediction](https://mlsanity.com/paper/2303.10895/) (2023)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2104.01528/) (2021)
- [Remember Intentions: Retrospective-Memory-based Trajectory Prediction](https://mlsanity.com/paper/2203.11474/) (2022)
- [Stepwise Goal-Driven Networks for Trajectory Prediction](https://mlsanity.com/paper/2103.14107/) (2021)
- [Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesis](https://mlsanity.com/paper/2103.07854/) (2021)
- [View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrums](https://mlsanity.com/paper/2110.07288/) (2021)
- [ADAPT: Efficient Multi-Agent Trajectory Prediction with Adaptation](https://mlsanity.com/paper/2307.14187/) (2023)
- [Goal-driven Self-Attentive Recurrent Networks for Trajectory Prediction](https://mlsanity.com/paper/2204.11561/) (2022)
- [SocialCircle: Learning the Angle-based Social Interaction Representation for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2310.05370/) (2023)
- [SMEMO: Social Memory for Trajectory Forecasting](https://mlsanity.com/paper/2203.12446/) (2022)
- [You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction](https://mlsanity.com/paper/2110.05304/) (2021)
- [MARC: Multipolicy and Risk-aware Contingency Planning for Autonomous Driving](https://mlsanity.com/paper/2308.12021/) (2023)

## Built on by (most cited first, 1 of 1)

- [Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review](https://mlsanity.com/paper/2506.14831/) (2025)

## Abstract (arXiv)

Analyzing and forecasting trajectories of agents like pedestrians plays a pivotal role for embodied intelligent applications. The inherent indeterminacy of human behavior and complex social interaction among a rich variety of agents make this task more challenging than common time-series forecasting. In this letter, we aim to explore a distinct formulation for multi-agent trajectory prediction framework. Specifically, we proposed a patching-based temporal feature extraction module and a graph-based social feature extraction module, enabling effective feature extraction and cross-scenario generalization. Moreover, we reassess the role of social interaction and present a novel method based on explicit modality modulation to integrate temporal and social features, thereby constructing an efficient single-stage inference pipeline. Results on public benchmark datasets demonstrate the superior performance of our model compared with the state-of-the-art methods. The code is available at: github.com/TIB-K330/pmm-net.