# SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents

arXiv: https://arxiv.org/abs/2402.17339 · HTML: https://mlsanity.com/paper/2402.17339/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 44 references that are in the map.
- First posted on arXiv 27 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints](https://mlsanity.com/paper/1806.01482/) (2018)
- [It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction](https://mlsanity.com/paper/2004.02025/) (2020)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting](https://mlsanity.com/paper/2012.01526/) (2020)
- [GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoning](https://mlsanity.com/paper/2204.08770/) (2022)
- [SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2104.01528/) (2021)
- [SocialVAE: Human Trajectory Prediction using Timewise Latents](https://mlsanity.com/paper/2203.08207/) (2022)
- [Human Trajectory Prediction via Neural Social Physics](https://mlsanity.com/paper/2207.10435/) (2022)
- [MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Prediction](https://mlsanity.com/paper/2201.07189/) (2022)
- [Interpretable Social Anchors for Human Trajectory Forecasting in Crowds](https://mlsanity.com/paper/2105.03136/) (2021)
- [Social Interpretable Tree for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2205.13296/) (2022)
- [Human Trajectory Forecasting with Explainable Behavioral Uncertainty](https://mlsanity.com/paper/2307.01817/) (2023)
- [Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for Trajectory Prediction](https://mlsanity.com/paper/2110.15016/) (2021)
- [Heter-Sim: Heterogeneous multi-agent systems simulation by interactive data-driven optimization](https://mlsanity.com/paper/1812.00307/) (2018)

## Built on by (most cited first, 9 of 9)

- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [Large-Scale Multi-Character Interaction Synthesis](https://mlsanity.com/paper/2505.14087/) (2025)
- [Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review](https://mlsanity.com/paper/2506.14831/) (2025)
- [Learning Extremely High Density Crowds as Active Matters](https://mlsanity.com/paper/2503.12168/) (2025)
- [Diffusion^2: Dual Diffusion Model with Uncertainty-Aware Adaptive Noise for Momentary Trajectory Prediction](https://mlsanity.com/paper/2510.04365/) (2025)
- [Eye Gaze-Informed and Context-Aware Pedestrian Trajectory Prediction in Shared Spaces with Automated Shuttles: A Virtual Reality Study](https://mlsanity.com/paper/2603.19812/) (2026)
- [Sparse Compositional Flow Matching by geometric assembly from motion primitives](https://mlsanity.com/paper/2605.23341/) (2026)
- [MamMA: A Mamba-Based Pedestrian Trajectory Prediction Algorithm Considering Occupancy Map and Pedestrian Awareness States](https://mlsanity.com/paper/2609.08041/) (2026)
- [MoCAR: Motion-code Coordinate-aware AutoRegression for Continuous Trajectory Forecasting](https://mlsanity.com/paper/2610.06210/) (2026)

## Abstract (arXiv)

Pedestrian trajectory prediction is the key technology in many applications for providing insights into human behavior and anticipating human future motions. Most existing empirical models are explicitly formulated by observed human behaviors using explicable mathematical terms with a deterministic nature, while recent work has focused on developing hybrid models combined with learning-based techniques for powerful expressiveness while maintaining explainability. However, the deterministic nature of the learned steering behaviors from the empirical models limits the models' practical performance. To address this issue, this work proposes the social conditional variational autoencoder (SocialCVAE) for predicting pedestrian trajectories, which employs a CVAE to explore behavioral uncertainty in human motion decisions. SocialCVAE learns socially reasonable motion randomness by utilizing a socially explainable interaction energy map as the CVAE's condition, which illustrates the future occupancy of each pedestrian's local neighborhood area. The energy map is generated using an energy-based interaction model, which anticipates the energy cost (i.e., repulsion intensity) of pedestrians' interactions with neighbors. Experimental results on two public benchmarks including 25 scenes demonstrate that SocialCVAE significantly improves prediction accuracy compared with the state-of-the-art methods, with up to 16.85% improvement in Average Displacement Error (ADE) and 69.18% improvement in Final Displacement Error (FDE).