# Are socially-aware trajectory prediction models really socially-aware?

arXiv: https://arxiv.org/abs/2108.10879 · HTML: https://mlsanity.com/paper/2108.10879/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 24 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints](https://mlsanity.com/paper/1806.01482/) (2018)
- [End-to-end Interpretable Neural Motion Planner](https://mlsanity.com/paper/2101.06679/) (2021)
- [It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction](https://mlsanity.com/paper/2004.02025/) (2020)
- [Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks](https://mlsanity.com/paper/1907.03395/) (2019)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016)
- [Social Ways: Learning Multi-Modal Distributions of Pedestrian Trajectories with GANs](https://mlsanity.com/paper/1904.09507/) (2019)
- [Adversarial Attacks on Deep Neural Networks for Time Series Classification](https://mlsanity.com/paper/1903.07054/) (2019)
- [Adversarial Attacks on Time Series](https://mlsanity.com/paper/1902.10755/) (2019)
- [Imperceptible Adversarial Attacks on Tabular Data](https://mlsanity.com/paper/1911.03274/) (2019)
- [Adversarial Attack on Skeleton-based Human Action Recognition](https://mlsanity.com/paper/1909.06500/) (2019)
- [Context-Aware Trajectory Prediction](https://mlsanity.com/paper/1705.02503/) (2017)
- [Hold me tight! Influence of discriminative features on deep network boundaries](https://mlsanity.com/paper/2002.06349/) (2020)
- [Injecting Knowledge in Data-driven Vehicle Trajectory Predictors](https://mlsanity.com/paper/2103.04854/) (2021)
- [An Evaluation of Trajectory Prediction Approaches and Notes on the TrajNet Benchmark](https://mlsanity.com/paper/1805.07663/) (2018)
- [Pedestrian Intention Prediction: A Multi-task Perspective](https://mlsanity.com/paper/2010.10270/) (2020)
- [Social-WaGDAT: Interaction-aware Trajectory Prediction via Wasserstein Graph Double-Attention Network](https://mlsanity.com/paper/2002.06241/) (2020)
- [SafeCritic: Collision-Aware Trajectory Prediction](https://mlsanity.com/paper/1910.06673/) (2019)
- [MonStereo: When Monocular and Stereo Meet at the Tail of 3D Human Localization](https://mlsanity.com/paper/2008.10913/) (2020)

## Built on by (most cited first, 30 of 32)

- [On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles](https://mlsanity.com/paper/2201.05057/) (2022)
- [UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction](https://mlsanity.com/paper/2403.15098/) (2024)
- [Traj-MAE: Masked Autoencoders for Trajectory Prediction](https://mlsanity.com/paper/2303.06697/) (2023)
- [Vehicle trajectory prediction works, but not everywhere](https://mlsanity.com/paper/2112.03909/) (2021)
- [A generic diffusion-based approach for 3D human pose prediction in the wild](https://mlsanity.com/paper/2210.05669/) (2022)
- [Social-Transmotion: Promptable Human Trajectory Prediction](https://mlsanity.com/paper/2312.16168/) (2023)
- [V2XPnP: Vehicle-to-Everything Spatio-Temporal Fusion for Multi-Agent Perception and Prediction](https://mlsanity.com/paper/2412.01812/) (2024)
- [Deep Interactive Motion Prediction and Planning: Playing Games with Motion Prediction Models](https://mlsanity.com/paper/2204.02392/) (2022)
- [CausalAgents: A Robustness Benchmark for Motion Forecasting using Causal Relationships](https://mlsanity.com/paper/2207.03586/) (2022)
- [Joint Metrics Matter: A Better Standard for Trajectory Forecasting](https://mlsanity.com/paper/2305.06292/) (2023)
- [Safety-compliant Generative Adversarial Networks for Human Trajectory Forecasting](https://mlsanity.com/paper/2209.12243/) (2022)
- [Motion Style Transfer: Modular Low-Rank Adaptation for Deep Motion Forecasting](https://mlsanity.com/paper/2211.03165/) (2022)
- [Pedestrian 3D Bounding Box Prediction](https://mlsanity.com/paper/2206.14195/) (2022)
- [Toward Reliable Human Pose Forecasting with Uncertainty](https://mlsanity.com/paper/2304.06707/) (2023)
- [Certified Human Trajectory Prediction](https://mlsanity.com/paper/2403.13778/) (2024)
- [Social-Pose: Enhancing Trajectory Prediction with Human Body Pose](https://mlsanity.com/paper/2507.22742/) (2025)
- [JRDB-Traj: A Dataset and Benchmark for Trajectory Forecasting in Crowds](https://mlsanity.com/paper/2311.02736/) (2023)
- [Sim-to-Real Causal Transfer: A Metric Learning Approach to Causally-Aware Interaction Representations](https://mlsanity.com/paper/2312.04540/) (2023)
- [Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness](https://mlsanity.com/paper/2403.14641/) (2024)
- [Realistic Adversarial Attacks for Robustness Evaluation of Trajectory Prediction Models via Future State Perturbation](https://mlsanity.com/paper/2505.06134/) (2025)
- [SVG-Net: An SVG-based Trajectory Prediction Model](https://mlsanity.com/paper/2110.03706/) (2021)
- [A Shared Representation for Photorealistic Driving Simulators](https://mlsanity.com/paper/2112.05134/) (2021)
- [INTENT: Trajectory Prediction Framework with Intention-Guided Contrastive Clustering](https://mlsanity.com/paper/2503.04952/) (2025)
- [Consistent Valid Physically-Realizable Adversarial Attack against Crowd-flow Prediction Models](https://mlsanity.com/paper/2303.02669/) (2023)
- [Robustness Benchmark of Road User Trajectory Prediction Models for Automated Driving](https://mlsanity.com/paper/2304.01895/) (2023)
- [Improving Trajectory Prediction in Dynamic Multi-Agent Environment by Dropping Waypoints](https://mlsanity.com/paper/2309.17338/) (2023)
- [Enhancing Trajectory Prediction through Self-Supervised Waypoint Noise Prediction](https://mlsanity.com/paper/2312.09466/) (2023)
- [iMotion-LLM: Instruction-Conditioned Trajectory Generation](https://mlsanity.com/paper/2406.06211/) (2024)
- [HHI-Assist: A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario](https://mlsanity.com/paper/2509.10096/) (2025)
- [Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehicle](https://mlsanity.com/paper/2309.06313/) (2023)

## Abstract (arXiv)

Our field has recently witnessed an arms race of neural network-based trajectory predictors. While these predictors are at the core of many applications such as autonomous navigation or pedestrian flow simulations, their adversarial robustness has not been carefully studied. In this paper, we introduce a socially-attended attack to assess the social understanding of prediction models in terms of collision avoidance. An attack is a small yet carefully-crafted perturbations to fail predictors. Technically, we define collision as a failure mode of the output, and propose hard- and soft-attention mechanisms to guide our attack. Thanks to our attack, we shed light on the limitations of the current models in terms of their social understanding. We demonstrate the strengths of our method on the recent trajectory prediction models. Finally, we show that our attack can be employed to increase the social understanding of state-of-the-art models. The code is available online: https://s-attack.github.io/