Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty
Not on the map. Why?
Abstract (arXiv)
Progress towards advanced systems for assisted and autonomous driving is leveraging recent advances in recognition and segmentation methods. Yet, we are still facing challenges in bringing reliable driving to inner cities, as those are composed of highly dynamic scenes observed from a moving platform at considerable speeds. Anticipation becomes a key element in order to react timely and prevent accidents. In this paper we argue that it is necessary to predict at least 1 second and we thus propose a new model that jointly predicts ego motion and people trajectories over such large time horizons. We pay particular attention to modeling the uncertainty of our estimates arising from the non-deterministic nature of natural traffic scenes. Our experimental results show that it is indeed possible to predict people trajectories at the desired time horizons and that our uncertainty estimates are informative of the prediction error. We also show that both sequence modeling of trajectories as well as our novel method of long term odometry prediction are essential for best performance.
Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Nov 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 19; 2022: 14; 2023: 8; 2024: 9; 2025: 8; 2026: 5.
63 papers.
Sort:
By area:
- 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
- 2021Stepwise Goal-Driven Networks for Trajectory Predictioncited by 62
- 2021On Exposing the Challenging Long Tail in Future Prediction of Traffic Actorscited by 48
- 2021Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Predictioncited by 36
- 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
- 2024VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecastingcited by 35
- 2023SwinLSTM:Improving Spatiotemporal Prediction Accuracy using Swin Transformer and LSTMcited by 32
- 2022STRPM: A Spatiotemporal Residual Predictive Model for High-Resolution Video Predictioncited by 31
- 2022PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learningcited by 23
- 2022Interpretable Self-Aware Neural Networks for Robust Trajectory Predictioncited by 22
- 2021Is attention to bounding boxes all you need for pedestrian action prediction?cited by 21
- 2021Unlimited Neighborhood Interaction for Heterogeneous Trajectory Predictioncited by 17
38 more
- 2021Collaborative Uncertainty in Multi-Agent Trajectory Forecastingcited by 17
- 2021Egocentric Human Trajectory Forecasting with a Wearable Camera and Multi-Modal Fusioncited by 17
- 2022Action-based Contrastive Learning for Trajectory Predictioncited by 15
- 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
- 2021TrouSPI-Net: Spatio-temporal attention on parallel atrous convolutions and U-GRUs for skeletal pedestrian crossing predictioncited by 13
- 2023A Novel Benchmarking Paradigm and a Scale- and Motion-Aware Model for Egocentric Pedestrian Trajectory Predictioncited by 12
- 2022Assessing Cross-dataset Generalization of Pedestrian Crossing Predictorscited by 11
- 2022Pedestrian 3D Bounding Box Predictioncited by 11
- 2024PIP-Net: Pedestrian Intention Prediction in the Wildcited by 11
- 2022Deep Virtual-to-Real Distillation for Pedestrian Crossing Predictioncited by 10
- 2023Survey on video anomaly detection in dynamic scenes with moving camerascited by 10
- 2021NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecastingcited by 8
- 2022PeSOTIF: a Challenging Visual Dataset for Perception SOTIF Problems in Long-tail Traffic Scenarioscited by 8
- 2021Towards Robust Human Trajectory Prediction in Raw Videoscited by 7
- 2021Two-stage Rule-induction Visual Reasoning on RPMs with an Application to Video Predictioncited by 7
- 2022On-Board Pedestrian Trajectory Prediction Using Behavioral Featurescited by 5
- 2021Indoor Future Person Localization from an Egocentric Wearable Cameracited by 4
- 2021Pedestrian Trajectory Prediction via Spatial Interaction Transformer Networkcited by 4
- 2023Context-aware Pedestrian Trajectory Prediction with Multimodal Transformercited by 4
- 2024Towards more realistic human motion prediction with attention to motion coordinationcited by 4
- 2024GTransPDM: A Graph-embedded Transformer with Positional Decoupling for Pedestrian Crossing Intention Predictioncited by 4
- 2022Recurrent Encoder-Decoder Networks for Vessel Trajectory Prediction with Uncertainty Estimationcited by 3
- 2023Local and Global Contextual Features Fusion for Pedestrian Intention Predictioncited by 3
- 2023Sparse Pedestrian Character Learning for Trajectory Predictioncited by 3
- 2021Probabilistic Human Motion Prediction via A Bayesian Neural Networkcited by 2
- 2022SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learningcited by 2
- 2022Pyramidal Predictive Network: A Model for Visual-frame Prediction Based on Predictive Coding Theorycited by 2
- 2022Predictive Coding Based Multiscale Network with Encoder-Decoder LSTM for Video Predictioncited by 2
- 2023About latent roles in forecasting players in team sportscited by 2
- 2025ASTRA: A Scene-aware TRAnsformer-based model for trajectory predictioncited by 2
- 2025Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Reviewcited by 2
- 2025Seeing Beyond Frames: Zero-Shot Pedestrian Intention Prediction with Raw Temporal Video and Multimodal Cuescited by 2
- 20213D Head-Position Prediction in First-Person View by Considering Head Pose for Human-Robot Eye Contactcited by 1
- 2021Hierarchical Graph-Convolutional Variational AutoEncoding for Generative Modelling of Human Motioncited by 1
- 2023Fusion-GRU: A Deep Learning Model for Future Bounding Box Prediction of Traffic Agents in Risky Driving Videoscited by 1
- 2024Gating Syn-to-Real Knowledge for Pedestrian Crossing Prediction in Safe Drivingcited by 1
- 2024Socially-Informed Reconstruction for Pedestrian Trajectory Forecastingcited by 1
- 2025Autoregression-free video prediction using diffusion model for mitigating error propagationcited by 1
50 most cited of 63 All 63 (JSON)
- 2026Unified Vision-Centric Pedestrian Crossing Action Prediction via Adaptive Patch Projection and Proactive Spatial Rectification7 Sep 2026
- 2026WaveSFNet: A Wavelet-Based Codec and Spatial--Frequency Dual-Domain Gating Network for Spatiotemporal Prediction24 Mar 2026
- 2026Single-Eye View: Monocular Real-time Perception Package for Autonomous Driving22 Mar 2026
- 2026TrajMamba: An Ego-Motion-Guided Mamba Model for Pedestrian Trajectory Prediction from an Egocentric Perspective16 Mar 2026
- 2026PFGNet: A Fully Convolutional Frequency-Guided Peripheral Gating Network for Efficient Spatiotemporal Predictive Learning24 Feb 2026
- 2025Occlusion-Aware Diffusion Model for Pedestrian Intention Prediction2 Nov 2025
- 2025Adaptive Output Steps: FlexiSteps Network for Dynamic Trajectory Prediction25 Aug 2025
- 2025Seeing Beyond Frames: Zero-Shot Pedestrian Intention Prediction with Raw Temporal Video and Multimodal Cues25 Jul 2025
- 2025Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review13 Jun 2025
- 2025Autoregression-free video prediction using diffusion model for mitigating error propagation28 May 2025
- 2025Temporal-contextual Event Learning for Pedestrian Crossing Intent Prediction4 Apr 2025
- 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions5 Mar 2025
38 more
- 2025ASTRA: A Scene-aware TRAnsformer-based model for trajectory prediction16 Jan 2025
- 2024Socially-Informed Reconstruction for Pedestrian Trajectory Forecasting5 Dec 2024
- 2024Lightweight Stochastic Video Prediction via Hybrid Warping4 Dec 2024
- 2024GTransPDM: A Graph-embedded Transformer with Positional Decoupling for Pedestrian Crossing Intention Prediction30 Sep 2024
- 2024Gating Syn-to-Real Knowledge for Pedestrian Crossing Prediction in Safe Driving24 Aug 2024
- 2024A Multi-Stage Goal-Driven Network for Pedestrian Trajectory Prediction26 Jun 2024
- 2024Video Generation with Learned Action Prior20 Jun 2024
- 2024Towards more realistic human motion prediction with attention to motion coordination4 Apr 2024
- 2024VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecasting25 Mar 2024
- 2024PIP-Net: Pedestrian Intention Prediction in the Wild20 Feb 2024
- 2023Sparse Pedestrian Character Learning for Trajectory Prediction27 Nov 2023
- 2023A Novel Benchmarking Paradigm and a Scale- and Motion-Aware Model for Egocentric Pedestrian Trajectory Prediction16 Oct 2023
- 2023SwinLSTM:Improving Spatiotemporal Prediction Accuracy using Swin Transformer and LSTM19 Aug 2023
- 2023Survey on video anomaly detection in dynamic scenes with moving cameras14 Aug 2023
- 2023Fusion-GRU: A Deep Learning Model for Future Bounding Box Prediction of Traffic Agents in Risky Driving Videos12 Aug 2023
- 2023Context-aware Pedestrian Trajectory Prediction with Multimodal Transformer7 Jul 2023
- 2023Local and Global Contextual Features Fusion for Pedestrian Intention Prediction1 May 2023
- 2023About latent roles in forecasting players in team sports17 Apr 2023
- 2022Predictive Coding Based Multiscale Network with Encoder-Decoder LSTM for Video Prediction22 Dec 2022
- 2022Interpretable Self-Aware Neural Networks for Robust Trajectory Prediction16 Nov 2022
- 2022PeSOTIF: a Challenging Visual Dataset for Perception SOTIF Problems in Long-tail Traffic Scenarios7 Nov 2022
- 2022Deep Virtual-to-Real Distillation for Pedestrian Crossing Prediction2 Nov 2022
- 2022On-Board Pedestrian Trajectory Prediction Using Behavioral Features21 Oct 2022
- 2022PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learning14 Oct 2022
- 2022Pyramidal Predictive Network: A Model for Visual-frame Prediction Based on Predictive Coding Theory15 Aug 2022
- 2022Action-based Contrastive Learning for Trajectory Prediction18 Jul 2022
- 2022Pedestrian 3D Bounding Box Prediction28 Jun 2022
- 2022Recurrent Encoder-Decoder Networks for Vessel Trajectory Prediction with Uncertainty Estimation11 May 2022
- 2022SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning2 Apr 2022
- 2022STRPM: A Spatiotemporal Residual Predictive Model for High-Resolution Video Prediction30 Mar 2022
- 2022Diffusion Probabilistic Modeling for Video Generation16 Mar 2022
- 2022Assessing Cross-dataset Generalization of Pedestrian Crossing Predictors29 Jan 2022
- 2021Pedestrian Trajectory Prediction via Spatial Interaction Transformer Network13 Dec 2021
- 2021Hierarchical Graph-Convolutional Variational AutoEncoding for Generative Modelling of Human Motion24 Nov 2021
- 2021Two-stage Rule-induction Visual Reasoning on RPMs with an Application to Video Prediction24 Nov 2021
- 2021Egocentric Human Trajectory Forecasting with a Wearable Camera and Multi-Modal Fusion1 Nov 2021
- 2021Collaborative Uncertainty in Multi-Agent Trajectory Forecasting26 Oct 2021
- 2021Predicting decision-making in the future: Human versus Machine9 Oct 2021
50 newest of 63 All 63 (JSON)
- 2021Stepwise Goal-Driven Networks for Trajectory Predictioncited by 62
- 2021On Exposing the Challenging Long Tail in Future Prediction of Traffic Actorscited by 48
- 2021Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Predictioncited by 36
- 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
- 2023SwinLSTM:Improving Spatiotemporal Prediction Accuracy using Swin Transformer and LSTMcited by 32
- 2022STRPM: A Spatiotemporal Residual Predictive Model for High-Resolution Video Predictioncited by 31
- 2022PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learningcited by 23
- 2022Interpretable Self-Aware Neural Networks for Robust Trajectory Predictioncited by 22
- 2021Is attention to bounding boxes all you need for pedestrian action prediction?cited by 21
- 2021Unlimited Neighborhood Interaction for Heterogeneous Trajectory Predictioncited by 17
- 2021Collaborative Uncertainty in Multi-Agent Trajectory Forecastingcited by 17
- 2021Egocentric Human Trajectory Forecasting with a Wearable Camera and Multi-Modal Fusioncited by 17
12 most cited of 56 All 56 (JSON)
- 2026WaveSFNet: A Wavelet-Based Codec and Spatial--Frequency Dual-Domain Gating Network for Spatiotemporal Prediction24 Mar 2026
- 2026PFGNet: A Fully Convolutional Frequency-Guided Peripheral Gating Network for Efficient Spatiotemporal Predictive Learning24 Feb 2026
- 2025Adaptive Output Steps: FlexiSteps Network for Dynamic Trajectory Prediction25 Aug 2025
- 2025Seeing Beyond Frames: Zero-Shot Pedestrian Intention Prediction with Raw Temporal Video and Multimodal Cues25 Jul 2025
- 2025Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Review13 Jun 2025
- 2025Temporal-contextual Event Learning for Pedestrian Crossing Intent Prediction4 Apr 2025
- 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions5 Mar 2025
- 2025ASTRA: A Scene-aware TRAnsformer-based model for trajectory prediction16 Jan 2025
- 2024Socially-Informed Reconstruction for Pedestrian Trajectory Forecasting5 Dec 2024
- 2024Lightweight Stochastic Video Prediction via Hybrid Warping4 Dec 2024
- 2024GTransPDM: A Graph-embedded Transformer with Positional Decoupling for Pedestrian Crossing Intention Prediction30 Sep 2024
- 2024Gating Syn-to-Real Knowledge for Pedestrian Crossing Prediction in Safe Driving24 Aug 2024
12 newest of 56 All 56 (JSON)