# Holistic Graph-based Motion Prediction

arXiv: https://arxiv.org/abs/2301.13545 · HTML: https://mlsanity.com/paper/2301.13545/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 38 references that are in the map.
- First posted on arXiv 31 Jan 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Heterogeneous Graph Attention Network](https://mlsanity.com/paper/1903.07293/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Heterogeneous Graph Transformer](https://mlsanity.com/paper/2003.01332/) (2020)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification](https://mlsanity.com/paper/2009.03509/) (2020)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings](https://mlsanity.com/paper/1905.01296/) (2019)
- [Scene Transformer: A unified architecture for predicting multiple agent trajectories](https://mlsanity.com/paper/2106.08417/) (2021)
- [A Survey on Heterogeneous Graph Embedding: Methods, Techniques, Applications and Sources](https://mlsanity.com/paper/2011.14867/) (2020)
- [Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions](https://mlsanity.com/paper/1906.08945/) (2019)
- [HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encoding](https://mlsanity.com/paper/2205.09753/) (2022)
- [Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving](https://mlsanity.com/paper/1808.05819/) (2018)
- [Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecasting](https://mlsanity.com/paper/1910.03650/) (2019)
- [Spectral Temporal Graph Neural Network for Trajectory Prediction](https://mlsanity.com/paper/2106.02930/) (2021)
- [Graph-Based Spatial-Temporal Convolutional Network for Vehicle Trajectory Prediction in Autonomous Driving](https://mlsanity.com/paper/2109.12764/) (2021)
- [ReCoG: A Deep Learning Framework with Heterogeneous Graph for Interaction-Aware Trajectory Prediction](https://mlsanity.com/paper/2012.05032/) (2020)

## Built on by (most cited first, 10 of 10)

- [SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphs](https://mlsanity.com/paper/2404.19379/) (2024)
- [nuScenes Knowledge Graph -- A comprehensive semantic representation of traffic scenes for trajectory prediction](https://mlsanity.com/paper/2312.09676/) (2023)
- [RedMotion: Motion Prediction via Redundancy Reduction](https://mlsanity.com/paper/2306.10840/) (2023)
- [Adversarial Backdoor Attack by Naturalistic Data Poisoning on Trajectory Prediction in Autonomous Driving](https://mlsanity.com/paper/2306.15755/) (2023)
- [KI-PMF: Knowledge Integrated Plausible Motion Forecasting](https://mlsanity.com/paper/2310.12007/) (2023)
- [SocialFormer: Social Interaction Modeling with Edge-enhanced Heterogeneous Graph Transformers for Trajectory Prediction](https://mlsanity.com/paper/2405.03809/) (2024)
- [Heterogeneous Graph-based Trajectory Prediction using Local Map Context and Social Interactions](https://mlsanity.com/paper/2311.18553/) (2023)
- [MAP-Former: Multi-Agent-Pair Gaussian Joint Prediction](https://mlsanity.com/paper/2404.19283/) (2024)
- [Goal-based Trajectory Prediction for improved Cross-Dataset Generalization](https://mlsanity.com/paper/2507.18196/) (2025)
- [Boundary-Guided Trajectory Prediction for Road Aware and Physically Feasible Autonomous Driving](https://mlsanity.com/paper/2505.06740/) (2025)

## Abstract (arXiv)

Motion prediction for automated vehicles in complex environments is a difficult task that is to be mastered when automated vehicles are to be used in arbitrary situations. Many factors influence the future motion of traffic participants starting with traffic rules and reaching from the interaction between each other to personal habits of human drivers. Therefore we present a novel approach for a graph-based prediction based on a heterogeneous holistic graph representation that combines temporal information, properties and relations between traffic participants as well as relations with static elements like the road network. The information are encoded through different types of nodes and edges that both are enriched with arbitrary features. We evaluated the approach on the INTERACTION and the Argoverse dataset and conducted an informative ablation study to demonstrate the benefit of different types of information for the motion prediction quality.