mlsanityData to 7 Oct 2026

Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generation

arXiv 2408.00766 ↗First posted 1 Aug 2024cs.CVcited by 18

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Diffusion models are promising for joint trajectory prediction and controllable generation in autonomous driving, but they face challenges of inefficient inference steps and high computational demands. To tackle these challenges, we introduce Optimal Gaussian Diffusion (OGD) and Estimated Clean Manifold (ECM) Guidance. OGD optimizes the prior distribution for a small diffusion time T and starts the reverse diffusion process from it. ECM directly injects guidance gradients to the estimated clean manifold, eliminating extensive gradient backpropagation throughout the network. Our methodology streamlines the generative process, enabling practical applications with reduced computational overhead. Experimental validation on the large-scale Argoverse 2 dataset demonstrates our approach's superior performance, offering a viable solution for computationally efficient, high-quality joint trajectory prediction and controllable generation for autonomous driving. Our project webpage is at https://yixiaowang7.github.io/OptTrajDiffPage/.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 46 of its 52 references that are in the map. First posted on arXiv 1 Aug 2024.

Builds on

46 of 52 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Joint Pedestrian Trajectory Prediction through Posterior Sampling30 Mar
  2. 2023SceneDM: Scene-level Multi-agent Trajectory Generation with Consistent Diffusion Models27 Nov
  3. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusion4 Apr
  4. 2022Guided Conditional Diffusion for Controllable Traffic Simulation31 Oct

Other references · 42

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  9. 2023Consistency Modelscited by 1,778
  10. 2021Variational Diffusion Modelscited by 1,145
  11. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  12. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
30 more
  1. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  2. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  3. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  4. 2020TNT: Target-driveN Trajectory Predictioncited by 345
  5. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 306
  6. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 292
  7. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 279
  8. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  9. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  10. 2021TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviorscited by 192
  11. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 184
  12. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Qualitycited by 181
  13. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  14. 2023MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Queryingcited by 139
  15. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  16. 2021Noise Estimation for Generative Diffusion Modelscited by 94
  17. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  18. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  19. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72
  20. 2022Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoderscited by 42
  21. 2022FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphscited by 41
  22. 2022How Much is Enough? A Study on Diffusion Times in Score-based Generative Modelscited by 35
  23. 2023QCNeXt: A Next-Generation Framework For Joint Multi-Agent Trajectory Predictioncited by 31
  24. 2023Dynamic Scenario Representation Learning for Motion Forecasting with Heterogeneous Graph Convolutional Recurrent Networkscited by 27
  25. 2023SEPT: Towards Efficient Scene Representation Learning for Motion Predictioncited by 27
  26. 2022PreTraM: Self-Supervised Pre-training via Connecting Trajectory and Mapcited by 25
  27. 2021Diverse Critical Interaction Generation for Planning and Planner Evaluationcited by 15
  28. 2023DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Cloudscited by 14
  29. 2024Q-SLAM: Quadric Representations for Monocular SLAMcited by 11
  30. 2022Domain Knowledge Driven Pseudo Labels for Interpretable Goal-Conditioned Interactive Trajectory Predictioncited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 8; 2026: 6.

18 papers.

Sort:

By area:

  1. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  2. 2025TrajDiff: End-to-end Autonomous Driving without Perception Annotationcited by 8
  3. 2025Continuous Locomotive Crowd Behavior Generationcited by 7
  4. 2024Hypergraph-based Motion Generation with Multi-modal Interaction Relational Reasoningcited by 6
  5. 2025Generative Active Learning for Long-tail Trajectory Prediction via Controllable Diffusion Modelcited by 6
  6. 2025Projected Coupled Diffusion for Test-Time Constrained Joint Generationcited by 6
  7. 2025Unified Human Localization and Trajectory Prediction with Monocular Visioncited by 4
  8. 2026ParkDiffusion++: Ego Intention Conditioned Joint Multi-Agent Trajectory Prediction for Automated Parking using Diffusion Modelscited by 3
  9. 2024Solving Prior Distribution Mismatch in Diffusion Models via Optimal Transportcited by 2
  10. 2024TrafficGamer: Reliable and Flexible Traffic Simulation for Safety-Critical Scenarios with Game-Theoretic Oraclescited by 1
  11. 2025EP-Diffuser: An Efficient Diffusion Model for Traffic Scene Generation and Prediction via Polynomial Representationscited by 1
  12. 2025SocialTraj: Two-Stage Socially-Aware Trajectory Prediction for Autonomous Driving via Conditional Diffusion Modelcited by 0
6 more
  1. 2025Path Diffuser: Diffusion Model for Data-Driven Traffic Simulatorcited by 0
  2. 2026Manifold-Optimal Guidance: A Unified Riemannian Control View of Diffusion Guidancecited by 0
  3. 2026ECTraj: Enhanced Consistency Training for Multi-Agent Trajectory Predictioncited by 0
  4. 2026Guiding Neuro-Symbolic Scenario Generation with Spatio-Temporal Logiccited by 0
  5. 2026Guided Diffusion with Distilled Vision-Language Reliability for Aerial Navigationcited by 0
  6. 2026Proposal-Conditioned Latent Diffusion for Closed-Loop Traffic Scenario Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion-ES: Gradient-free Planning with Diffusion for Autonomous Driving and Zero-Shot Instruction Following
  2. 2024ADM: Accelerated Diffusion Model via Estimated Priors for Robust Motion Prediction under Uncertainties
  3. 2026G2DP: Diffusion Planning with Spatio-Temporal Grid Guidance
  4. 2024Intention-aware Denoising Diffusion Model for Trajectory Prediction
  5. 2023DICE: Diverse Diffusion Model with Scoring for Trajectory Prediction
  6. 2026HAD: Combining Hierarchical Diffusion with Metric-Decoupled RL for End-to-End Driving
  7. 2025DiffRefiner: Coarse to Fine Trajectory Planning via Diffusion Refinement with Semantic Interaction for End to End Autonomous Driving
  8. 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Driving