mlsanityData to 14 Oct 2026

KEMP: Keyframe-Based Hierarchical End-to-End Deep Model for Long-Term Trajectory Prediction

arXiv 2205.04624 ↗First posted 10 May 2022cs.CV cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Predicting future trajectories of road agents is a critical task for autonomous driving. Recent goal-based trajectory prediction methods, such as DenseTNT and PECNet, have shown good performance on prediction tasks on public datasets. However, they usually require complicated goal-selection algorithms and optimization. In this work, we propose KEMP, a hierarchical end-to-end deep learning framework for trajectory prediction. At the core of our framework is keyframe-based trajectory prediction, where keyframes are representative states that trace out the general direction of the trajectory. KEMP first predicts keyframes conditioned on the road context, and then fills in intermediate states conditioned on the keyframes and the road context. Under our general framework, goal-conditioned methods are special cases in which the number of keyframes equal to one. Unlike goal-conditioned methods, our keyframe predictor is learned automatically and does not require hand-crafted goal-selection algorithms. We evaluate our model on public benchmarks and our model ranked 1st on Waymo Open Motion Dataset Leaderboard (as of September 1, 2021).

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 30 of its 38 references that are in the map. First posted on arXiv 10 May 2022.

Builds on

30 of 38 references.

References · 30

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
  5. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  6. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 618
  7. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  8. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  9. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  10. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  11. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 397
  12. 2020TNT: Target-driveN Trajectory Predictioncited by 347
18 more
  1. 2017DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agentscited by 310
  2. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  3. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  4. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  5. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 236
  6. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 179
  7. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  8. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  9. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 113
  10. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  11. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  12. 2021Learning to Predict Vehicle Trajectories with Model-based Planningcited by 63
  13. 2018Learning Plannable Representations with Causal InfoGANcited by 57
  14. 2018How would surround vehicles move? A Unified Framework for Maneuver Classification and Motion Predictioncited by 46
  15. 2019Learning Robotic Manipulation through Visual Planning and Actingcited by 41
  16. 2019Predicting Motion of Vulnerable Road Users using High-Definition Maps and Efficient ConvNetscited by 16
  17. 2020Fast Risk Assessment for Autonomous Vehicles Using Learned Models of Agent Futurescited by 9
  18. 2021Convex Risk Bounded Continuous-Time Trajectory Planning in Uncertain Nonconvex Environmentscited by 5

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 6; 2024: 2; 2025: 2.

11 papers.

  1. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 156
  2. 2022GANet: Goal Area Network for Motion Forecastingcited by 49
  3. 2023The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Reviewcited by 29
  4. 2024SemanticFormer: Holistic and Semantic Traffic Scene Representation for Trajectory Prediction using Knowledge Graphscited by 11
  5. 2023Bridging the Gap Between Multi-Step and One-Shot Trajectory Prediction via Self-Supervisioncited by 7
  6. 2024Safety Implications of Explainable Artificial Intelligence in End-to-End Autonomous Drivingcited by 7
  7. 2025Foresight in Motion: Reinforcing Trajectory Prediction with Reward Heuristicscited by 6
  8. 2023A Hierarchical Hybrid Learning Framework for Multi-agent Trajectory Predictioncited by 3
  9. 2023Real-Time Tube-Based Non-Gaussian Risk Bounded Motion Planning for Stochastic Nonlinear Systems in Uncertain Environments via Motion Primitivescited by 0
  10. 2023Convex Risk Bounded Continuous-Time Trajectory Planning and Tube Design in Uncertain Nonconvex Environmentscited by 0
  11. 2025HyperDiff: Hypergraph Guided Diffusion Model for 3D Human Pose Estimationcited by 0