mlsanityData to 7 Oct 2026

Trajectory Prediction with Latent Belief Energy-Based Model

arXiv 2104.03086 ↗First posted 7 Apr 2021cs.LGcited by 60

Not on the map. Why?

Abstract (arXiv)

Human trajectory prediction is critical for autonomous platforms like self-driving cars or social robots. We present a latent belief energy-based model (LB-EBM) for diverse human trajectory forecast. LB-EBM is a probabilistic model with cost function defined in the latent space to account for the movement history and social context. The low-dimensionality of the latent space and the high expressivity of the EBM make it easy for the model to capture the multimodality of pedestrian trajectory distributions. LB-EBM is learned from expert demonstrations (i.e., human trajectories) projected into the latent space. Sampling from or optimizing the learned LB-EBM yields a belief vector which is used to make a path plan, which then in turn helps to predict a long-range trajectory. The effectiveness of LB-EBM and the two-step approach are supported by strong empirical results. Our model is able to make accurate, multi-modal, and social compliant trajectory predictions and improves over prior state-of-the-arts performance on the Stanford Drone trajectory prediction benchmark by 10.9% and on the ETH-UCY benchmark by 27.6%.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 44 of its 68 references that are in the map. First posted on arXiv 7 Apr 2021.

Builds on

44 of 68 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2014Conditional Generative Adversarial Netscited by 2,152
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  7. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  8. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  9. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 324
  10. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  11. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 296
  12. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
32 more
  1. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 235
  2. 2019Human Motion Trajectory Prediction: A Surveycited by 231
  3. 2019Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networkscited by 228
  4. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  5. 2017Social Attention: Modeling Attention in Human Crowdscited by 185
  6. 2020Transformer Networks for Trajectory Forecastingcited by 180
  7. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 178
  8. 2019Multiple Futures Predictioncited by 178
  9. 2016A Theory of Generative ConvNetcited by 174
  10. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  11. 2019Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Modelcited by 147
  12. 2019Multi-Agent Tensor Fusion for Contextual Trajectory Predictioncited by 131
  13. 2019Peeking into the Future: Predicting Future Person Activities and Locations in Videoscited by 112
  14. 2019SR-LSTM: State Refinement for LSTM towards Pedestrian Trajectory Predictioncited by 111
  15. 2020Learning Latent Space Energy-Based Prior Modelcited by 104
  16. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  17. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 92
  18. 2018Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMscited by 77
  19. 2019Conditional Generative Neural System for Probabilistic Trajectory Predictioncited by 76
  20. 2019The Garden of Forking Paths: Towards Multi-Future Trajectory Predictioncited by 72
  21. 2020Trajectory Forecasts in Unknown Environments Conditioned on Grid-Based Planscited by 72
  22. 2018How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Predictioncited by 46
  23. 2019Conditional Flow Variational Autoencoders for Structured Sequence Predictioncited by 36
  24. 2017CAR-Net: Clairvoyant Attentive Recurrent Networkcited by 34
  25. 2019Analyzing the Variety Loss in the Context of Probabilistic Trajectory Predictioncited by 30
  26. 2020Joint Training of Variational Auto-Encoder and Latent Energy-Based Modelcited by 30
  27. 2019Implicit Generation and Generalization in Energy-Based Modelscited by 21
  28. 2018MX-LSTM: mixing tracklets and vislets to jointly forecast trajectories and head posescited by 17
  29. 2019Stochastic trajectory prediction with social graph networkcited by 9
  30. 2020Learning Latent Space Energy-Based Prior Model for Molecule Generationcited by 6
  31. 2020Semi-supervised Learning by Latent Space Energy-Based Model of Symbol-Vector Couplingcited by 3
  32. 2020SSP: Single Shot Future Trajectory Predictioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 13; 2023: 17; 2024: 14; 2025: 7; 2026: 3.

60 papers.

Sort:

By area:

  1. 2022Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformercited by 211
  2. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 184
  3. 2023Leapfrog Diffusion Model for Stochastic Trajectory Predictioncited by 129
  4. 2022Latent Diffusion Energy-Based Model for Interpretable Text Modelingcited by 67
  5. 2021Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Predictioncited by 57
  6. 2022HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequencescited by 57
  7. 2022Adaptive Trajectory Prediction via Transferable GNNcited by 51
  8. 2021View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrumscited by 44
  9. 2023EigenTrajectory: Low-Rank Descriptors for Multi-Modal Trajectory Forecastingcited by 41
  10. 2024SingularTrajectory: Universal Trajectory Predictor Using Diffusion Modelcited by 35
  11. 2023FEND: A Future Enhanced Distribution-Aware Contrastive Learning Framework for Long-tail Trajectory Predictioncited by 31
  12. 2024Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Predictioncited by 31
38 more
  1. 2022Dynamic-Group-Aware Networks for Multi-Agent Trajectory Prediction with Relational Reasoningcited by 28
  2. 2023SocialCircle: Learning the Angle-based Social Interaction Representation for Pedestrian Trajectory Predictioncited by 27
  3. 2022SMEMO: Social Memory for Trajectory Forecastingcited by 26
  4. 2022GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Modelcited by 25
  5. 2024LG-Traj: LLM Guided Pedestrian Trajectory Predictioncited by 24
  6. 2022End-to-End Trajectory Distribution Prediction Based on Occupancy Grid Mapscited by 22
  7. 2021MSN: Multi-Style Network for Trajectory Predictioncited by 19
  8. 2021SSAGCN: Social Soft Attention Graph Convolution Network for Pedestrian Trajectory Predictioncited by 19
  9. 2024S2AC: Energy-Based Reinforcement Learning with Stein Soft Actor Criticcited by 18
  10. 2021Latent Space Energy-Based Model of Symbol-Vector Coupling for Text Generation and Classificationcited by 14
  11. 2023Another Vertical View: A Hierarchical Network for Heterogeneous Trajectory Prediction via Spectrumscited by 14
  12. 2023Learning Energy-Based Prior Model with Diffusion-Amortized MCMCcited by 13
  13. 2022D2-TPred: Discontinuous Dependency for Trajectory Prediction under Traffic Lightscited by 10
  14. 2022Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encodingcited by 8
  15. 2024"Noisier" Noise Contrastive Eestimation is (Almost) Maximum Likelihoodcited by 8
  16. 2023A Tale of Two Latent Flows: Learning Latent Space Normalizing Flow with Short-run Langevin Flow for Approximate Inferencecited by 7
  17. 2025Continuous Locomotive Crowd Behavior Generationcited by 7
  18. 2023Diverse and Faithful Knowledge-Grounded Dialogue Generation via Sequential Posterior Inferencecited by 6
  19. 2024Multi-Agent Inverse Reinforcement Learning in Real World Unstructured Pedestrian Crowdscited by 6
  20. 2024SocialCircle+: Learning the Angle-based Conditioned Interaction Representation for Pedestrian Trajectory Predictioncited by 6
  21. 2023AdapTraj: A Multi-Source Domain Generalization Framework for Multi-Agent Trajectory Predictioncited by 5
  22. 2024Hitchhiker's guide on the relation of Energy-Based Models with other generative models, sampling and statistical physics: a comprehensive reviewcited by 5
  23. 2021Hierarchical Motion Encoder-Decoder Network for Trajectory Forecastingcited by 4
  24. 2024Latent Space Energy-based Neural ODEscited by 4
  25. 2024TrajDiffuse: A Conditional Diffusion Model for Environment-Aware Trajectory Predictioncited by 4
  26. 2022An Energy-Based Prior for Generative Saliencycited by 3
  27. 2023Data-Driven Stochastic Motion Evaluation and Optimization with Image by Spatially-Aligned Temporal Encodingcited by 3
  28. 2023Temporal DINO: A Self-supervised Video Strategy to Enhance Action Predictioncited by 3
  29. 2023Sparse Pedestrian Character Learning for Trajectory Predictioncited by 3
  30. 2025Stochastic Trajectory Prediction under Unstructured Constraintscited by 3
  31. 2025Generative AI for Autonomous Driving: A Reviewcited by 3
  32. 2023Likelihood-Based Generative Radiance Field with Latent Space Energy-Based Model for 3D-Aware Disentangled Image Representationcited by 2
  33. 2024Modeling Pedestrian Intrinsic Uncertainty for Multimodal Stochastic Trajectory Prediction via Energy Plan Denoisingcited by 2
  34. 2024Trajectory Forecasting through Low-Rank Adaptation of Discrete Latent Codescited by 2
  35. 2025ASTRA: A Scene-aware TRAnsformer-based model for trajectory predictioncited by 2
  36. 2025Learning Energy-Based Generative Models via Potential Flow: A Variational Principle Approach to Probability Density Homotopy Matchingcited by 2
  37. 2025Recent Advances in Multi-Agent Human Trajectory Prediction: A Comprehensive Reviewcited by 2
  38. 2022TPAD: Identifying Effective Trajectory Predictions Under the Guidance of Trajectory Anomaly Detection Modelcited by 1

50 most cited of 60 All 60 (JSON)