# CASPNet++: Joint Multi-Agent Motion Prediction

arXiv: https://arxiv.org/abs/2308.07751 · HTML: https://mlsanity.com/paper/2308.07751/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Builds on (most cited first, 27 of 27)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks](https://mlsanity.com/paper/1809.10732/) (2018)
- [CoverNet: Multimodal Behavior Prediction using Trajectory Sets](https://mlsanity.com/paper/1911.10298/) (2019)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [Convolutional Social Pooling for Vehicle Trajectory Prediction](https://mlsanity.com/paper/1805.06771/) (2018)
- [GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2109.01827/) (2021)
- [Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Prediction](https://mlsanity.com/paper/2104.00563/) (2021)
- [HOME: Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2105.10968/) (2021)
- [Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals](https://mlsanity.com/paper/2106.15004/) (2021)
- [Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving](https://mlsanity.com/paper/1808.05819/) (2018)
- [What-If Motion Prediction for Autonomous Driving](https://mlsanity.com/paper/2008.10587/) (2020)
- [Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans](https://mlsanity.com/paper/2001.00735/) (2020)
- [Occupancy Flow Fields for Motion Forecasting in Autonomous Driving](https://mlsanity.com/paper/2203.03875/) (2022)
- [Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective](https://mlsanity.com/paper/1806.07772/) (2018)
- [Leveraging Future Relationship Reasoning for Vehicle Trajectory Prediction](https://mlsanity.com/paper/2305.14715/) (2023)
- [Graph Neural Networks for Modelling Traffic Participant Interaction](https://mlsanity.com/paper/1903.01254/) (2019)
- [PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation](https://mlsanity.com/paper/2109.11094/) (2021)
- [SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction](https://mlsanity.com/paper/2007.13078/) (2020)
- [Context-Aware Scene Prediction Network (CASPNet)](https://mlsanity.com/paper/2201.06933/) (2022)

## Built on by (most cited first, 4 of 4)

- [Motion Forecasting via Model-Based Risk Minimization](https://mlsanity.com/paper/2409.10585/) (2024)
- [CASPFormer: Trajectory Prediction from BEV Images with Deformable Attention](https://mlsanity.com/paper/2409.17790/) (2024)
- [AI-Powered CPS-Enabled Vulnerable-User-Aware Urban Transportation Digital Twin: Methods and Applications](https://mlsanity.com/paper/2501.10396/) (2024)
- [LMFormer: Lane based Motion Prediction Transformer](https://mlsanity.com/paper/2504.10275/) (2025)

## Abstract (arXiv)

The prediction of road users' future motion is a critical task in supporting advanced driver-assistance systems (ADAS). It plays an even more crucial role for autonomous driving (AD) in enabling the planning and execution of safe driving maneuvers. Based on our previous work, Context-Aware Scene Prediction Network (CASPNet), an improved system, CASPNet++, is proposed. In this work, we focus on further enhancing the interaction modeling and scene understanding to support the joint prediction of all road users in a scene using spatiotemporal grids to model future occupancy. Moreover, an instance-based output head is introduced to provide multi-modal trajectories for agents of interest. In extensive quantitative and qualitative analysis, we demonstrate the scalability of CASPNet++ in utilizing and fusing diverse environmental input sources such as HD maps, Radar detection, and Lidar segmentation. Tested on the urban-focused prediction dataset nuScenes, CASPNet++ reaches state-of-the-art performance. The model has been deployed in a testing vehicle, running in real-time with moderate computational resources.