# Learning Sparse Interaction Graphs of Partially Detected Pedestrians for Trajectory Prediction

arXiv: https://arxiv.org/abs/2107.07056 · HTML: https://mlsanity.com/paper/2107.07056/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [DropEdge: Towards Deep Graph Convolutional Networks on Node Classification](https://mlsanity.com/paper/1907.10903/) (2019)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models](https://mlsanity.com/paper/1802.08773/) (2018)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [Human Motion Trajectory Prediction: A Survey](https://mlsanity.com/paper/1905.06113/) (2019)
- [Learning Discrete Structures for Graph Neural Networks](https://mlsanity.com/paper/1903.11960/) (2019)
- [Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognition](https://mlsanity.com/paper/1904.12659/) (2019)
- [Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2005.08514/) (2020)
- [Social Attention: Modeling Attention in Human Crowds](https://mlsanity.com/paper/1710.04689/) (2017)
- [Crowd-Robot Interaction: Crowd-aware Robot Navigation with Attention-based Deep Reinforcement Learning](https://mlsanity.com/paper/1809.08835/) (2018)
- [Causal Discovery with Reinforcement Learning](https://mlsanity.com/paper/1906.04477/) (2019)
- [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017)
- [Human Trajectory Forecasting in Crowds: A Deep Learning Perspective](https://mlsanity.com/paper/2007.03639/) (2020)
- [SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Prediction](https://mlsanity.com/paper/1903.02793/) (2019)
- [Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data](https://mlsanity.com/paper/2001.03093/) (2020)
- [Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2011.04820/) (2020)
- [3DOF Pedestrian Trajectory Prediction Learned from Long-Term Autonomous Mobile Robot Deployment Data](https://mlsanity.com/paper/1710.00126/) (2017)
- [Long-term Pedestrian Trajectory Prediction using Mutable Intention Filter and Warp LSTM](https://mlsanity.com/paper/2007.00113/) (2020)
- [Online monitoring for safe pedestrian-vehicle interactions](https://mlsanity.com/paper/1910.05599/) (2019)

## Built on by (most cited first, 10 of 10)

- [Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph](https://mlsanity.com/paper/2203.01821/) (2022)
- [SoNIC: Safe Social Navigation with Adaptive Conformal Inference and Constrained Reinforcement Learning](https://mlsanity.com/paper/2407.17460/) (2024)
- [Towards Generalizable Safety in Crowd Navigation via Conformal Uncertainty Handling](https://mlsanity.com/paper/2508.05634/) (2025)
- [Attention Graph for Multi-Robot Social Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2401.17914/) (2024)
- [Transformable Gaussian Reward Function for Socially-Aware Navigation with Deep Reinforcement Learning](https://mlsanity.com/paper/2402.14569/) (2024)
- [CoNav: A Benchmark for Human-Centered Collaborative Navigation](https://mlsanity.com/paper/2406.02425/) (2024)
- [ART: Adaptive Relational Transformer for Pedestrian Trajectory Prediction with Temporal-Aware Relations](https://mlsanity.com/paper/2604.03649/) (2026)
- [Belief Aided Navigation using Bayesian Reinforcement Learning for Avoiding Humans in Blind Spots](https://mlsanity.com/paper/2403.10105/) (2024)
- [A Spatio-temporal Graph Network Allowing Incomplete Trajectory Input for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2501.13973/) (2025)
- [MamMA: A Mamba-Based Pedestrian Trajectory Prediction Algorithm Considering Occupancy Map and Pedestrian Awareness States](https://mlsanity.com/paper/2609.08041/) (2026)

## Abstract (arXiv)

Multi-pedestrian trajectory prediction is an indispensable element of autonomous systems that safely interact with crowds in unstructured environments. Many recent efforts in trajectory prediction algorithms have focused on understanding social norms behind pedestrian motions. Yet we observe these works usually hold two assumptions, which prevent them from being smoothly applied to robot applications: (1) positions of all pedestrians are consistently tracked, and (2) the target agent pays attention to all pedestrians in the scene. The first assumption leads to biased interaction modeling with incomplete pedestrian data. The second assumption introduces aggregation of redundant surrounding information, and the target agent may be affected by unimportant neighbors or present overly conservative motion. Thus, we propose Gumbel Social Transformer, in which an Edge Gumbel Selector samples a sparse interaction graph of partially detected pedestrians at each time step. A Node Transformer Encoder and a Masked LSTM encode pedestrian features with sampled sparse graphs to predict trajectories. We demonstrate that our model overcomes potential problems caused by the aforementioned assumptions, and our approach outperforms related works in trajectory prediction benchmarks. Code is available at urlhttps://github.com/tedhuang96/gst.