mlsanityData to 14 Oct 2026

DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

arXiv 1704.04394 ↗First posted 14 Apr 2017cs.CVcited by 310

Not on the map. Why?

Abstract (arXiv)

We introduce a Deep Stochastic IOC RNN Encoderdecoder framework, DESIRE, for the task of future predictions of multiple interacting agents in dynamic scenes. DESIRE effectively predicts future locations of objects in multiple scenes by 1) accounting for the multi-modal nature of the future prediction (i.e., given the same context, future may vary), 2) foreseeing the potential future outcomes and make a strategic prediction based on that, and 3) reasoning not only from the past motion history, but also from the scene context as well as the interactions among the agents. DESIRE achieves these in a single end-to-end trainable neural network model, while being computationally efficient. The model first obtains a diverse set of hypothetical future prediction samples employing a conditional variational autoencoder, which are ranked and refined by the following RNN scoring-regression module. Samples are scored by accounting for accumulated future rewards, which enables better long-term strategic decisions similar to IOC frameworks. An RNN scene context fusion module jointly captures past motion histories, the semantic scene context and interactions among multiple agents. A feedback mechanism iterates over the ranking and refinement to further boost the prediction accuracy. We evaluate our model on two publicly available datasets: KITTI and Stanford Drone Dataset. Our experiments show that the proposed model significantly improves the prediction accuracy compared to other baseline methods.

Key facts. Cited by 310 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 89; 2022: 73; 2023: 68; 2024: 36; 2025: 24; 2026: 19.

310 papers.

Sort:

By area:

  1. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  2. 2021AgentFormer: Agent-Aware Transformers for Socio-Temporal Multi-Agent Forecastingcited by 331
  3. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  4. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 287
  5. 2022Wayformer: Motion Forecasting via Simple & Efficient Attention Networkscited by 280
  6. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  7. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  8. 2021Multimodal Motion Prediction with Stacked Transformerscited by 208
  9. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  10. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 185
  11. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
  12. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
38 more
  1. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 156
  2. 2022Model-Based Imitation Learning for Urban Drivingcited by 150
  3. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  4. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 138
  5. 2022Learning from All Vehiclescited by 135
  6. 2021Latent Variable Sequential Set Transformers For Joint Multi-Agent Motion Predictioncited by 131
  7. 2022GroupNet: Multiscale Hypergraph Neural Networks for Trajectory Prediction with Relational Reasoningcited by 128
  8. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  9. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 117
  10. 2021LaneRCNN: Distributed Representations for Graph-Centric Motion Forecastingcited by 113
  11. 2021Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversalscited by 113
  12. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 110
  13. 2021Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Taskscited by 101
  14. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  15. 2023EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoningcited by 97
  16. 2022Remember Intentions: Retrospective-Memory-based Trajectory Predictioncited by 86
  17. 2023End-to-End Vectorized HD-map Construction with Piecewise Bezier Curvecited by 82
  18. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  19. 2023HumanMAC: Masked Motion Completion for Human Motion Predictioncited by 81
  20. 2021Explainability of deep vision-based autonomous driving systems: Review and challengescited by 80
  21. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  22. 2021SimNet: Learning Reactive Self-driving Simulations from Real-world Observationscited by 77
  23. 2021LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agentscited by 64
  24. 2022ScePT: Scene-consistent, Policy-based Trajectory Predictions for Planningcited by 64
  25. 2021Learning to Predict Vehicle Trajectories with Model-based Planningcited by 63
  26. 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
  27. 2021Stepwise Goal-Driven Networks for Trajectory Predictioncited by 62
  28. 2021MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Predictioncited by 61
  29. 2022Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulationcited by 61
  30. 2021Trajectory Prediction with Latent Belief Energy-Based Modelcited by 60
  31. 2022MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Predictioncited by 59
  32. 2021LOKI: Long Term and Key Intentions for Trajectory Predictioncited by 57
  33. 2023TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Predictioncited by 56
  34. 2023LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraintscited by 53
  35. 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
  36. 2022GANet: Goal Area Network for Motion Forecastingcited by 49
  37. 2021On Exposing the Challenging Long Tail in Future Prediction of Traffic Actorscited by 48
  38. 2023Leveraging Future Relationship Reasoning for Vehicle Trajectory Predictioncited by 48

50 most cited of 310 All 310 (JSON)