mlsanityData to 7 Oct 2026

Diverse Yet Consistent: Context-Guided Diffusion with Energy-Based Joint Refinement for Multi-Agent Motion Prediction

arXiv 2605.22017 ↗First posted 21 May 2026cs.CVcited by 0

On the map: Diffusion models → Video generation

Abstract (arXiv)

Deepgenerative models havebecomeapromisingapproach for human motion prediction due to their ability to capture multimodal distributions and represent diverse human be haviors. However, generating predictions that are both di verse and jointly consistent among interacting agents re mains challenging. In addition, most existing approaches are primarily evaluated using single-agent (marginal) met rics, which fail to fully reflect the joint dynamics of multi agent interactions. We propose a diffusion-based frame work that improves multi-agent motion prediction by lever aging rich contextual information from historical trajecto ries. This information is incorporated through a guidance mechanism to enhance the diversity and expressiveness of predicted motions. To further enforce interaction consis tency, we introduce an energy-based formulation that re fines the joint trajectory distribution while preserving the plausibility of individual trajectories. Extensive experi ments on four benchmark datasets demonstrate that our approach consistently outperforms existing methods. No tably, our approach substantially improves both marginal (ADE/FDE) and joint (JADE/JFDE) metrics on ETH/UCY over strong marginal baselines. Compared with prior joint prediction methods, it delivers significant gains in marginal metrics while maintaining competitive joint performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 50 of its 68 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

50 of 68 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022PhysDiff: Physics-Guided Human Motion Diffusion Model5 Dec

Other references · 49

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  5. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  6. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 327
  7. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 324
  8. 2017A Structured Self-attentive Sentence Embeddingcited by 318
  9. 2023An Edit Friendly DDPM Noise Space: Inversion and Manipulationscited by 243
  10. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 235
  11. 2018Convolutional Social Pooling for Vehicle Trajectory Predictioncited by 221
  12. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
37 more
  1. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 184
  2. 2020Transformer Networks for Trajectory Forecastingcited by 180
  3. 2020From Goals, Waypoints & Paths To Long Term Human Trajectory Forecastingcited by 166
  4. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 137
  5. 2019Multi-Agent Tensor Fusion for Contextual Trajectory Predictioncited by 131
  6. 2023Leapfrog Diffusion Model for Stochastic Trajectory Predictioncited by 129
  7. 2020Human Trajectory Forecasting in Crowds: A Deep Learning Perspectivecited by 128
  8. 2022GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoningcited by 128
  9. 2023EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoningcited by 97
  10. 2021LookOut: Diverse Multi-Future Prediction and Planning for Self-Drivingcited by 96
  11. 2016An Uncertain Future: Forecasting from Static Images using Variational Autoencoderscited by 86
  12. 2022Remember Intentions: Retrospective-Memory-based Trajectory Predictioncited by 86
  13. 2020PiP: Planning-informed Trajectory Prediction for Autonomous Drivingcited by 80
  14. 2022SocialVAE: Human Trajectory Prediction using Timewise Latentscited by 65
  15. 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
  16. 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
  17. 2021MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Predictioncited by 48
  18. 2022Non-Probability Sampling Network for Stochastic Human Trajectory Predictioncited by 46
  19. 2021View Vertically: A Hierarchical Network for Trajectory Prediction via Fourier Spectrumscited by 44
  20. 2023EigenTrajectory: Low-Rank Descriptors for Multi-Modal Trajectory Forecastingcited by 41
  21. 2023Social-Transmotion: Promptable Human Trajectory Predictioncited by 33
  22. 2025MoFlow: One-Step Flow Matching for Human Trajectory Forecasting via Implicit Maximum Likelihood Estimation based Distillationcited by 31
  23. 2019Analyzing the Variety Loss in the Context of Probabilistic Trajectory Predictioncited by 30
  24. 2023Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensorscited by 29
  25. 2024Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioningcited by 29
  26. 2022Pose Forecasting in Industrial Human-Robot Collaborationcited by 24
  27. 2023Joint Metrics Matter: A Better Standard for Trajectory Forecastingcited by 17
  28. 2024Progressive Pretext Task Learning for Human Trajectory Predictioncited by 16
  29. 2022Forecasting Human Trajectory from Scene Historycited by 14
  30. 2020GraphTCN: Spatio-Temporal Interaction Modeling for Human Trajectory Predictioncited by 12
  31. 2024EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modelingcited by 11
  32. 2022Self-Supervised Point Cloud Registration with Deep Versatile Descriptorscited by 10
  33. 2023Timestamp-supervised Wearable-based Activity Segmentation and Recognition with Contrastive Learning and Order-Preserving Optimal Transportcited by 10
  34. 2024Joint Pedestrian Trajectory Prediction through Posterior Samplingcited by 10
  35. 2025Multi-modal Knowledge Distillation-based Human Trajectory Forecastingcited by 10
  36. 2023JRDB-Traj: A Dataset and Benchmark for Trajectory Forecasting in Crowdscited by 4
  37. 2024Context-Conditioned Spatio-Temporal Predictive Learning for Reliable V2V Channel Predictioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024MDMP: Multi-modal Diffusion for supervised Motion Predictions with uncertainty
  2. 2024Predicting Long-Term Human Behaviors in Discrete Representations via Physics-Guided Diffusion
  3. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions
  4. 2026Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Prediction
  5. 2024COLLAGE: Collaborative Human-Agent Interaction Generation using Hierarchical Latent Diffusion and Language Models
  6. 2025Diffusion Forcing for Multi-Agent Interaction Sequence Modeling
  7. 2024Multi-Condition Latent Diffusion Network for Scene-Aware Neural Human Motion Prediction
  8. 2024Bayesian-Optimized One-Step Diffusion Model with Knowledge Distillation for Real-Time 3D Human Motion Prediction