mlsanityData to 14 Oct 2026

MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying

arXiv 2306.17770 ↗First posted 30 Jun 2023cs.CVcited by 140

Not on the map. Why?

Cited by 39 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Motion prediction is crucial for autonomous driving systems to understand complex driving scenarios and make informed decisions. However, this task is challenging due to the diverse behaviors of traffic participants and complex environmental contexts. In this paper, we propose Motion TRansformer (MTR) frameworks to address these challenges. The initial MTR framework utilizes a transformer encoder-decoder structure with learnable intention queries, enabling efficient and accurate prediction of future trajectories. By customizing intention queries for distinct motion modalities, MTR improves multimodal motion prediction while reducing reliance on dense goal candidates. The framework comprises two essential processes: global intention localization, identifying the agent's intent to enhance overall efficiency, and local movement refinement, adaptively refining predicted trajectories for improved accuracy. Moreover, we introduce an advanced MTR++ framework, extending the capability of MTR to simultaneously predict multimodal motion for multiple agents. MTR++ incorporates symmetric context modeling and mutually-guided intention querying modules to facilitate future behavior interaction among multiple agents, resulting in scene-compliant future trajectories. Extensive experimental results demonstrate that the MTR framework achieves state-of-the-art performance on the highly-competitive motion prediction benchmarks, while the MTR++ framework surpasses its precursor, exhibiting enhanced performance and efficiency in predicting accurate multimodal future trajectories for multiple agents.

Key facts. Cited by 140 papers in the map (data through 14 Oct 2026). Builds on 55 of its 74 references that are in the map. First posted on arXiv 30 Jun 2023.

Builds on

55 of 74 references.

References · 55

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  7. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  8. 2017Non-local Neural Networkscited by 2,293
  9. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  10. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  11. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  12. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
38 more
  1. 2021Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Datasetcited by 537
  2. 2020VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representationcited by 530
  3. 2021Conditional DETR for Fast Training Convergencecited by 424
  4. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  5. 2020Learning Lane Graph Representations for Motion Forecastingcited by 382
  6. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 381
  7. 2020TNT: Target-driveN Trajectory Predictioncited by 347
  8. 2021DenseTNT: End-to-end Trajectory Prediction from Dense Goal Setscited by 295
  9. 2022Motion Transformer with Global Intention Localization and Local Movement Refinementcited by 270
  10. 2019CoverNet: Multimodal Behavior Prediction using Trajectory Setscited by 259
  11. 2021MultiPath++: Efficient Information Fusion and Trajectory Aggregation for Behavior Predictioncited by 246
  12. 2020It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Predictioncited by 236
  13. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 212
  14. 2021Multimodal Motion Prediction with Stacked Transformerscited by 208
  15. 2019PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settingscited by 179
  16. 2019Multiple Futures Predictioncited by 178
  17. 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 167
  18. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  19. 2021GOHOME: Graph-Oriented Heatmap Output for future Motion Estimationcited by 146
  20. 2021HOME: Heatmap Output for future Motion Estimationcited by 122
  21. 2023DSVT: Dynamic Sparse Voxel Transformer with Rotated Setscited by 122
  22. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  23. 2020Implicit Latent Variable Model for Scene-Consistent Motion Forecastingcited by 116
  24. 2022HDGT: Heterogeneous Driving Graph Transformer for Multi-Agent Trajectory Prediction via Scene Encodingcited by 110
  25. 2019Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Datacited by 104
  26. 2021THOMAS: Trajectory Heatmap Output with learned Multi-Agent Samplingcited by 101
  27. 2021TPCN: Temporal Point Cloud Networks for Motion Forecastingcited by 94
  28. 2018Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Drivingcited by 90
  29. 2019Multi-Head Attention for Multi-Modal Joint Vehicle Motion Forecastingcited by 87
  30. 2022M2I: From Factored Marginal Trajectory Prediction to Interactive Predictioncited by 81
  31. 2020TPNet: Trajectory Proposal Network for Motion Predictioncited by 80
  32. 2020MANTRA: Memory Augmented Networks for Multiple Trajectory Predictioncited by 69
  33. 2020Diverse and Admissible Trajectory Forecasting through Multimodal Context Understandingcited by 68
  34. 2022MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detectioncited by 65
  35. 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
  36. 2021Identifying Driver Interactions via Conditional Behavior Predictioncited by 48
  37. 2022A Unified Query-based Paradigm for Point Cloud Understandingcited by 32
  38. 2020PRANK: motion Prediction based on RANKingcited by 22

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 52; 2025: 51; 2026: 29.

140 papers.

Sort:

By area:

  1. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  2. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 231
  3. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  4. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  5. 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mambacited by 87
  6. 2023Trajeglish: Traffic Modeling as Next-Token Predictioncited by 60
  7. 2024BehaviorGPT: Smart Agent Simulation for Autonomous Driving with Next-Patch Predictioncited by 50
  8. 2024SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rolloutcited by 48
  9. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 47
  10. 2023DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Drivingcited by 46
  11. 2024Gen-Drive: Enhancing Diffusion Generative Driving Policies with Reward Modeling and Reinforcement Learning Fine-tuningcited by 42
  12. 2024Solving Motion Planning Tasks with a Scalable Generative Modelcited by 40
38 more
  1. 2024Closed-Loop Supervised Fine-Tuning of Tokenized Traffic Modelscited by 40
  2. 2024WcDT: World-centric Diffusion Transformer for Traffic Scene Generationcited by 32
  3. 2024V2XPnP: Vehicle-to-Everything Spatio-Temporal Fusion for Multi-Agent Perception and Predictioncited by 31
  4. 2024DriveGPT: Scaling Autoregressive Behavior Models for Drivingcited by 29
  5. 2024SIMPL: A Simple and Efficient Multi-agent Motion Prediction Baseline for Autonomous Drivingcited by 27
  6. 2024Improving Agent Behaviors with RL Fine-tuning for Autonomous Drivingcited by 27
  7. 2024CMP: Cooperative Motion Prediction with Multi-Agent Communicationcited by 25
  8. 2024SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Predictioncited by 25
  9. 2023Open-sourced Data Ecosystem in Autonomous Driving: the Present and Futurecited by 23
  10. 2025GuideFlow: Constraint-Guided Flow Matching for Planning in End-to-End Autonomous Drivingcited by 22
  11. 2024Promptable Closed-loop Traffic Simulationcited by 20
  12. 2025Scaling Laws of Motion Forecasting and Planning -- Technical Reportcited by 19
  13. 2024Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generationcited by 18
  14. 2025GaussRender: Learning 3D Occupancy with Gaussian Renderingcited by 18
  15. 2025Trajectory Mamba: Efficient Attention-Mamba Forecasting Model Based on Selective SSMcited by 18
  16. 2023EDA: Evolving and Distinct Anchors for Multimodal Motion Predictioncited by 17
  17. 2024RoadPainter: Points Are Ideal Navigators for Topology transformERcited by 17
  18. 2025DriveAction: A Benchmark for Exploring Human-like Driving Decisions in VLA Modelscited by 17
  19. 2025DIVER: Reinforced Diffusion Breaks Imitation Bottlenecks in End-to-End Autonomous Drivingcited by 17
  20. 2025Co-MTP: A Cooperative Trajectory Prediction Framework with Multi-Temporal Fusion for Autonomous Drivingcited by 14
  21. 2025Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directionscited by 14
  22. 2025Trajectory Prediction Meets Large Language Models: A Surveycited by 14
  23. 2025Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuningcited by 14
  24. 2024Large Language Models Powered Context-aware Motion Prediction in Autonomous Drivingcited by 13
  25. 2024AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Drivingcited by 13
  26. 2025UniMM: A Unified Mixture Model Framework for Multi-Agent Simulationcited by 13
  27. 2024CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous Drivingcited by 12
  28. 2024Planning with Adaptive World Models for Autonomous Drivingcited by 11
  29. 2024Motion Forecasting in Continuous Drivingcited by 11
  30. 2025LANGTRAJ: Diffusion Model and Dataset for Language-Conditioned Trajectory Simulationcited by 11
  31. 2025QuantV2X: A Fully Quantized Multi-Agent System for Cooperative Perceptioncited by 11
  32. 2023Deception Game: Closing the Safety-Learning Loop in Interactive Robot Autonomycited by 10
  33. 2023MGTR: Multi-Granular Transformer for Motion Prediction with LiDARcited by 9
  34. 2024Improving Out-of-Distribution Generalization of Trajectory Prediction for Autonomous Driving via Polynomial Representationscited by 9
  35. 2025Motion Forecasting for Autonomous Vehicles: A Surveycited by 8
  36. 2025TrajFlow: Multi-modal Motion Prediction via Flow Matchingcited by 8
  37. 2024JointMotion: Joint Self-Supervision for Joint Motion Predictioncited by 7
  38. 2024Risk-Aware Vehicle Trajectory Prediction Under Safety-Critical Scenarioscited by 7

50 most cited of 140 All 140 (JSON)