# Peeking into the Future: Predicting Future Person Activities and Locations in Videos

arXiv: https://arxiv.org/abs/1902.03748 · HTML: https://mlsanity.com/paper/1902.03748/

## Key facts

- Cited by 114 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2019.

## Built on by (most cited first, 30 of 114)

- [Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos](https://mlsanity.com/paper/2204.01696/) (2022)
- [SGCN:Sparse Graph Convolution Network for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2104.01528/) (2021)
- [Remember Intentions: Retrospective-Memory-based Trajectory Prediction](https://mlsanity.com/paper/2203.11474/) (2022)
- [Object-Region Video Transformers](https://mlsanity.com/paper/2110.06915/) (2021)
- [Human Trajectory Prediction via Neural Social Physics](https://mlsanity.com/paper/2207.10435/) (2022)
- [Stepwise Goal-Driven Networks for Trajectory Prediction](https://mlsanity.com/paper/2103.14107/) (2021)
- [Trajectory Prediction with Latent Belief Energy-Based Model](https://mlsanity.com/paper/2104.03086/) (2021)
- [Adaptive Trajectory Prediction via Transferable GNN](https://mlsanity.com/paper/2203.05046/) (2022)
- [Three Steps to Multimodal Trajectory Prediction: Modality Clustering, Classification and Synthesis](https://mlsanity.com/paper/2103.07854/) (2021)
- [Non-Probability Sampling Network for Stochastic Human Trajectory Prediction](https://mlsanity.com/paper/2203.13471/) (2022)
- [Imitation Learning for Human Pose Prediction](https://mlsanity.com/paper/1909.03449/) (2019)
- [Interpretable Social Anchors for Human Trajectory Forecasting in Crowds](https://mlsanity.com/paper/2105.03136/) (2021)
- [Learning Pedestrian Group Representations for Multi-modal Trajectory Prediction](https://mlsanity.com/paper/2207.09953/) (2022)
- [AGENT: A Benchmark for Core Psychological Reasoning](https://mlsanity.com/paper/2102.12321/) (2021)
- [A Dynamic Spatial-temporal Attention Network for Early Anticipation of Traffic Accidents](https://mlsanity.com/paper/2106.10197/) (2021)
- [Human Trajectory Prediction via Counterfactual Analysis](https://mlsanity.com/paper/2107.14202/) (2021)
- [EigenTrajectory: Low-Rank Descriptors for Multi-Modal Trajectory Forecasting](https://mlsanity.com/paper/2307.09306/) (2023)
- [Learning to Anticipate Egocentric Actions by Imagination](https://mlsanity.com/paper/2101.04924/) (2021)
- [Graph-based Spatial Transformer with Memory Replay for Multi-future Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2206.05712/) (2022)
- [Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking](https://mlsanity.com/paper/2102.09117/) (2021)
- [SingularTrajectory: Universal Trajectory Predictor Using Diffusion Model](https://mlsanity.com/paper/2403.18452/) (2024)
- [Social-Transmotion: Promptable Human Trajectory Prediction](https://mlsanity.com/paper/2312.16168/) (2023)
- [Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction](https://mlsanity.com/paper/2112.15012/) (2021)
- [Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction](https://mlsanity.com/paper/2403.18447/) (2024)
- [Personalized Trajectory Prediction via Distribution Discrimination](https://mlsanity.com/paper/2107.14204/) (2021)
- [Rethinking Learning Approaches for Long-Term Action Anticipation](https://mlsanity.com/paper/2210.11566/) (2022)
- [SocialCircle: Learning the Angle-based Social Interaction Representation for Pedestrian Trajectory Prediction](https://mlsanity.com/paper/2310.05370/) (2023)
- [SMEMO: Social Memory for Trajectory Forecasting](https://mlsanity.com/paper/2203.12446/) (2022)
- [Is an Object-Centric Video Representation Beneficial for Transfer?](https://mlsanity.com/paper/2207.10075/) (2022)
- [PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learning](https://mlsanity.com/paper/2210.07886/) (2022)

## Abstract (arXiv)

Deciphering human behaviors to predict their future paths/trajectories and what they would do from videos is important in many applications. Motivated by this idea, this paper studies predicting a pedestrian's future path jointly with future activities. We propose an end-to-end, multi-task learning system utilizing rich visual features about human behavioral information and interaction with their surroundings. To facilitate the training, the network is learned with an auxiliary task of predicting future location in which the activity will happen. Experimental results demonstrate our state-of-the-art performance over two public benchmarks on future trajectory prediction. Moreover, our method is able to produce meaningful future activity prediction in addition to the path. The result provides the first empirical evidence that joint modeling of paths and activities benefits future path prediction.