# Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Prediction

arXiv: https://arxiv.org/abs/1910.09077 · HTML: https://mlsanity.com/paper/1910.09077/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2019.

## Built on by (most cited first, 25 of 25)

- [DEFT: Detection Embeddings for Tracking](https://mlsanity.com/paper/2102.02267/) (2021)
- [Predicting Pedestrian Crossing Intention with Feature Fusion and Spatio-Temporal Attention](https://mlsanity.com/paper/2104.05485/) (2021)
- [Behavioral Intention Prediction in Driving Scenes: A Survey](https://mlsanity.com/paper/2211.00385/) (2022)
- [PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learning](https://mlsanity.com/paper/2210.07886/) (2022)
- [VPTR: Efficient Transformers for Video Prediction](https://mlsanity.com/paper/2203.15836/) (2022)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [Temporally smooth online action detection using cycle-consistent future anticipation](https://mlsanity.com/paper/2104.08030/) (2021)
- [IntFormer: Predicting pedestrian intention with the aid of the Transformer architecture](https://mlsanity.com/paper/2105.08647/) (2021)
- [TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing prediction](https://mlsanity.com/paper/2109.00953/) (2021)
- [Assessing Cross-dataset Generalization of Pedestrian Crossing Predictors](https://mlsanity.com/paper/2201.12626/) (2022)
- [ERA: Expert Retrieval and Assembly for Early Action Prediction](https://mlsanity.com/paper/2207.09675/) (2022)
- [Comparison of Spatio-Temporal Models for Human Motion and Pose Forecasting in Face-to-Face Interaction Scenarios](https://mlsanity.com/paper/2203.03245/) (2022)
- [Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation](https://mlsanity.com/paper/2210.05991/) (2022)
- [Cross or Wait? Predicting Pedestrian Interaction Outcomes at Unsignalized Crossings](https://mlsanity.com/paper/2304.08260/) (2023)
- [WatchPed: Pedestrian Crossing Intention Prediction Using Embedded Sensors of Smartwatch](https://mlsanity.com/paper/2208.07441/) (2022)
- [DaDe: Delay-adaptive Detector for Streaming Perception](https://mlsanity.com/paper/2212.11558/) (2022)
- [Diving Deeper Into Pedestrian Behavior Understanding: Intention Estimation, Action Prediction, and Event Risk Assessment](https://mlsanity.com/paper/2407.00446/) (2024)
- [TrajFusionNet: Pedestrian Crossing Intention Prediction via Fusion of Sequential and Visual Trajectory Representations](https://mlsanity.com/paper/2508.19866/) (2025)
- [Intention-Aware Decision-Making for Mixed Intersection Scenarios](https://mlsanity.com/paper/2303.17493/) (2023)
- [Predicting and Analyzing Pedestrian Crossing Behavior at Unsignalized Crossings](https://mlsanity.com/paper/2404.09574/) (2024)
- [Gating Syn-to-Real Knowledge for Pedestrian Crossing Prediction in Safe Driving](https://mlsanity.com/paper/2409.06707/) (2024)
- [Motion Prediction Performance Analysis for Autonomous Driving Systems and the Effects of Tracking Noise](https://mlsanity.com/paper/2104.08368/) (2021)
- [MulCPred: Learning Multi-modal Concepts for Explainable Pedestrian Action Prediction](https://mlsanity.com/paper/2409.09446/) (2024)
- [Predicting Pedestrian Crossing Behavior in Germany and Japan: Insights into Model Transferability](https://mlsanity.com/paper/2412.03689/) (2024)
- [Evaluating Few-Shot Temporal Reasoning of LLMs for Human Activity Prediction in Smart Environments](https://mlsanity.com/paper/2602.11176/) (2026)

## Abstract (arXiv)

In this paper, we present an end-to-end future-prediction model that focuses on pedestrian safety. Specifically, our model uses previous video frames, recorded from the perspective of the vehicle, to predict if a pedestrian will cross in front of the vehicle. The long term goal of this work is to design a fully autonomous system that acts and reacts as a defensive human driver would --- predicting future events and reacting to mitigate risk. We focus on pedestrian-vehicle interactions because of the high risk of harm to the pedestrian if their actions are miss-predicted. Our end-to-end model consists of two stages: the first stage is an encoder/decoder network that learns to predict future video frames. The second stage is a deep spatio-temporal network that utilizes the predicted frames of the first stage to predict the pedestrian's future action. Our system achieves state-of-the-art accuracy on pedestrian behavior prediction and future frames prediction on the Joint Attention for Autonomous Driving (JAAD) dataset.