# Learning a Pedestrian Social Behavior Dictionary

arXiv: https://arxiv.org/abs/2212.01426 · HTML: https://mlsanity.com/paper/2212.01426/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 2 Dec 2022.

## Builds on (most cited first, 29 of 29)

- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecasting](https://mlsanity.com/paper/2103.14023/) (2021)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints](https://mlsanity.com/paper/1806.01482/) (2018)
- [It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction](https://mlsanity.com/paper/2004.02025/) (2020)
- [Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks](https://mlsanity.com/paper/1907.03395/) (2019)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [Transformer Networks for Trajectory Forecasting](https://mlsanity.com/paper/2003.08111/) (2020)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoning](https://mlsanity.com/paper/2204.08770/) (2022)
- [Learning Representations by Predicting Bags of Visual Words](https://mlsanity.com/paper/2002.12247/) (2020)
- [SocialVAE: Human Trajectory Prediction using Timewise Latents](https://mlsanity.com/paper/2203.08207/) (2022)
- [Recursive Social Behavior Graph for Trajectory Prediction](https://mlsanity.com/paper/2004.10402/) (2020)
- [MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Prediction](https://mlsanity.com/paper/2201.07189/) (2022)
- [Social NCE: Contrastive Learning of Socially-aware Motion Representations](https://mlsanity.com/paper/2012.11717/) (2020)
- [Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesis](https://mlsanity.com/paper/2103.07854/) (2021)
- [Deep TEN: Texture Encoding Network](https://mlsanity.com/paper/1612.02844/) (2016)
- [Interpretable Social Anchors for Human Trajectory Forecasting in Crowds](https://mlsanity.com/paper/2105.03136/) (2021)
- [Learning Pedestrian Group Representations for Multi-modal Trajectory Prediction](https://mlsanity.com/paper/2207.09953/) (2022)
- [Deep Texture Manifold for Ground Terrain Recognition](https://mlsanity.com/paper/1803.10896/) (2018)
- [Personalized Trajectory Prediction via Distribution Discrimination](https://mlsanity.com/paper/2107.14204/) (2021)
- [Goal-driven Self-Attentive Recurrent Networks for Trajectory Prediction](https://mlsanity.com/paper/2204.11561/) (2022)
- [Social and Scene-Aware Trajectory Prediction in Crowded Spaces](https://mlsanity.com/paper/1909.08840/) (2019)
- [AC-VRNN: Attentive Conditional-VRNN for Multi-Future Trajectory Prediction](https://mlsanity.com/paper/2005.08307/) (2020)
- [Unsupervised Learning of Video Representations via Dense Trajectory Clustering](https://mlsanity.com/paper/2006.15731/) (2020)
- [Pedestrian Motion Model Using Non-Parametric Trajectory Clustering and Discrete Transition Points](https://mlsanity.com/paper/2001.10571/) (2020)

## Built on by (most cited first, 1 of 1)

- [Socially Cognizant Robotics for a Technology Enhanced Society](https://mlsanity.com/paper/2310.18303/) (2023)

## Abstract (arXiv)

Understanding pedestrian behavior patterns is a key component to building autonomous agents that can navigate among humans. We seek a learned dictionary of pedestrian behavior to obtain a semantic description of pedestrian trajectories. Supervised methods for dictionary learning are impractical since pedestrian behaviors may be unknown a priori and the process of manually generating behavior labels is prohibitively time consuming. We instead utilize a novel, unsupervised framework to create a taxonomy of pedestrian behavior observed in a specific space. First, we learn a trajectory latent space that enables unsupervised clustering to create an interpretable pedestrian behavior dictionary. We show the utility of this dictionary for building pedestrian behavior maps to visualize space usage patterns and for computing the distributions of behaviors. We demonstrate a simple but effective trajectory prediction by conditioning on these behavior labels. While many trajectory analysis methods rely on RNNs or transformers, we develop a lightweight, low-parameter approach and show results comparable to SOTA on the ETH and UCY datasets.