# Human Trajectory Prediction via Neural Social Physics

arXiv: https://arxiv.org/abs/2207.10435 · HTML: https://mlsanity.com/paper/2207.10435/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 77 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 30 of 36)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Physics-informed neural networks (PINNs) for fluid mechanics: A review](https://mlsanity.com/paper/2105.09506/) (2021)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints](https://mlsanity.com/paper/1806.01482/) (2018)
- [It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction](https://mlsanity.com/paper/2004.02025/) (2020)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting](https://mlsanity.com/paper/2012.01526/) (2020)
- [Peeking into the Future: Predicting Future Person Activities and Locations in Videos](https://mlsanity.com/paper/1902.03748/) (2019)
- [SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2104.01528/) (2021)
- [Conditional Generative Neural System for Probabilistic Trajectory Prediction](https://mlsanity.com/paper/1905.01631/) (2019)
- [The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction](https://mlsanity.com/paper/1912.06445/) (2019)
- [Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Plans](https://mlsanity.com/paper/2001.00735/) (2020)
- [Social NCE: Contrastive Learning of Socially-aware Motion Representations](https://mlsanity.com/paper/2012.11717/) (2020)
- [Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contact](https://mlsanity.com/paper/2103.16021/) (2021)
- [Conditional Flow Variational Autoencoders for Structured Sequence Prediction](https://mlsanity.com/paper/1908.09008/) (2019)
- [High-order Differentiable Autoencoder for Nonlinear Model Reduction](https://mlsanity.com/paper/2102.11026/) (2021)
- [Context-Aware Trajectory Prediction](https://mlsanity.com/paper/1705.02503/) (2017)
- [MeshingNet: A New Mesh Generation Method based on Deep Learning](https://mlsanity.com/paper/2004.07016/) (2020)
- [CSCNet: Contextual Semantic Consistency Network for Trajectory Prediction in Crowded Spaces](https://mlsanity.com/paper/2202.08506/) (2022)
- [Fine-grained differentiable physics: a yarn-level model for fabrics](https://mlsanity.com/paper/2202.00504/) (2022)
- [Informative Scene Decomposition for Crowd Analysis, Comparison and Simulation Guidance](https://mlsanity.com/paper/2004.14107/) (2020)
- [LCollision: Fast Generation of Collision-Free Human Poses using Learned Non-Penetration Constraints](https://mlsanity.com/paper/2011.03632/) (2020)
- [Deep Social Force](https://mlsanity.com/paper/2109.12081/) (2021)
- [SimAug: Learning Robust Representations from Simulation for Trajectory Prediction](https://mlsanity.com/paper/2004.02022/) (2020)
- [Active Learning of Neural Collision Handler for Complex 3D Mesh Deformations](https://mlsanity.com/paper/2110.07727/) (2021)
- [Sliding Sequential CVAE with Time Variant Socially-aware Rethinking for Trajectory Prediction](https://mlsanity.com/paper/2110.15016/) (2021)

## Built on by (most cited first, 30 of 66)

- [EigenTrajectory: Low-Rank Descriptors for Multi-Modal Trajectory Forecasting](https://mlsanity.com/paper/2307.09306/) (2023)
- [SingularTrajectory: Universal Trajectory Predictor Using Diffusion Model](https://mlsanity.com/paper/2403.18452/) (2024)
- [Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction](https://mlsanity.com/paper/2403.18447/) (2024)
- [Fast Inference and Update of Probabilistic Density Estimation on Trajectory Prediction](https://mlsanity.com/paper/2308.08824/) (2023)
- [SocialCircle: Learning the Angle-based Social Interaction Representation for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2310.05370/) (2023)
- [T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memory](https://mlsanity.com/paper/2403.10052/) (2024)
- [Unsupervised Sampling Promoting for Stochastic Human Trajectory Prediction](https://mlsanity.com/paper/2304.04298/) (2023)
- [Joint Metrics Matter: A Better Standard for Trajectory Forecasting](https://mlsanity.com/paper/2305.06292/) (2023)
- [Progressive Pretext Task Learning for Human Trajectory Prediction](https://mlsanity.com/paper/2407.11588/) (2024)
- [Another Vertical View: A Hierarchical Network for Heterogeneous Trajectory Prediction via Spectrums](https://mlsanity.com/paper/2304.05106/) (2023)
- [Diffusion-Based Environment-Aware Trajectory Prediction](https://mlsanity.com/paper/2403.11643/) (2024)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [Evaluating Pedestrian Trajectory Prediction Methods with Respect to Autonomous Driving](https://mlsanity.com/paper/2308.05194/) (2023)
- [Social Physics Informed Diffusion Model for Crowd Simulation](https://mlsanity.com/paper/2402.06680/) (2024)
- [Intention-aware Denoising Diffusion Model for Trajectory Prediction](https://mlsanity.com/paper/2403.09190/) (2024)
- [SICNav-Diffusion: Safe and Interactive Crowd Navigation with Diffusion Trajectory Predictions](https://mlsanity.com/paper/2503.08858/) (2025)
- [Joint Pedestrian Trajectory Prediction through Posterior Sampling](https://mlsanity.com/paper/2404.00237/) (2024)
- [Physical Plausibility-aware Trajectory Prediction via Locomotion Embodiment](https://mlsanity.com/paper/2503.17267/) (2025)
- [Pishgu: Universal Path Prediction Network Architecture for Real-time Cyber-physical Edge Systems](https://mlsanity.com/paper/2210.08057/) (2022)
- [SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents](https://mlsanity.com/paper/2402.17339/) (2024)
- [Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision](https://mlsanity.com/paper/2506.06253/) (2025)
- [Human Trajectory Forecasting with Explainable Behavioral Uncertainty](https://mlsanity.com/paper/2307.01817/) (2023)
- [Robot Trajectron: Trajectory Prediction-based Shared Control for Robot Manipulation](https://mlsanity.com/paper/2402.02499/) (2024)
- [SocialCircle+: Learning the Angle-based Conditioned Interaction Representation for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2409.14984/) (2024)
- [Continuous Locomotive Crowd Behavior Generation](https://mlsanity.com/paper/2504.04756/) (2025)
- [SocialMOIF: Multi-Order Intention Fusion for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2504.15616/) (2025)
- [Staged Contact-Aware Global Human Motion Forecasting](https://mlsanity.com/paper/2309.08947/) (2023)
- [Goal-based Neural Physics Vehicle Trajectory Prediction Model](https://mlsanity.com/paper/2409.15182/) (2024)
- [T2FPV: Dataset and Method for Correcting First-Person View Errors in Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2209.11294/) (2022)
- [Benchmark for Models Predicting Human Behavior in Gap Acceptance Scenarios](https://mlsanity.com/paper/2211.05455/) (2022)

## Abstract (arXiv)

Trajectory prediction has been widely pursued in many fields, and many model-based and model-free methods have been explored. The former include rule-based, geometric or optimization-based models, and the latter are mainly comprised of deep learning approaches. In this paper, we propose a new method combining both methodologies based on a new Neural Differential Equation model. Our new model (Neural Social Physics or NSP) is a deep neural network within which we use an explicit physics model with learnable parameters. The explicit physics model serves as a strong inductive bias in modeling pedestrian behaviors, while the rest of the network provides a strong data-fitting capability in terms of system parameter estimation and dynamics stochasticity modeling. We compare NSP with 15 recent deep learning methods on 6 datasets and improve the state-of-the-art performance by 5.56%-70%. Besides, we show that NSP has better generalizability in predicting plausible trajectories in drastically different scenarios where the density is 2-5 times as high as the testing data. Finally, we show that the physics model in NSP can provide plausible explanations for pedestrian behaviors, as opposed to black-box deep learning. Code is available: https://github.com/realcrane/Human-Trajectory-Prediction-via-Neural-Social-Physics.