# M2I: From Factored Marginal Trajectory Prediction to Interactive Prediction

arXiv: https://arxiv.org/abs/2202.11884 · HTML: https://mlsanity.com/paper/2202.11884/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 45 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 30 of 41)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks](https://mlsanity.com/paper/1803.10892/) (2018)
- [Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset](https://mlsanity.com/paper/2104.10133/) (2021)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Prediction](https://mlsanity.com/paper/1910.05449/) (2019)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [TNT: Target-driveN Trajectory Prediction](https://mlsanity.com/paper/2008.08294/) (2020)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents](https://mlsanity.com/paper/1704.04394/) (2017)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networks](https://mlsanity.com/paper/1809.10732/) (2018)
- [It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction](https://mlsanity.com/paper/2004.02025/) (2020)
- [INTERACTION Dataset: An INTERnational, Adversarial and Cooperative moTION Dataset in Interactive Driving Scenarios with Semantic Maps](https://mlsanity.com/paper/1910.03088/) (2019)
- [Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks](https://mlsanity.com/paper/1907.03395/) (2019)
- [PRECOG: PREdiction Conditioned On Goals in Visual Multi-Agent Settings](https://mlsanity.com/paper/1905.01296/) (2019)
- [Multiple Futures Prediction](https://mlsanity.com/paper/1911.00997/) (2019)
- [Scene Transformer: A unified architecture for predicting multiple agent trajectories](https://mlsanity.com/paper/2106.08417/) (2021)
- [GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2109.01827/) (2021)
- [Multi-Agent Tensor Fusion for Contextual Trajectory Prediction](https://mlsanity.com/paper/1904.04776/) (2019)
- [HOME: Heatmap Output for future Motion Estimation](https://mlsanity.com/paper/2105.10968/) (2021)
- [Implicit Latent Variable Model for Scene-Consistent Motion Forecasting](https://mlsanity.com/paper/2007.12036/) (2020)
- [Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data](https://mlsanity.com/paper/1910.08233/) (2019)
- [Diverse Trajectory Forecasting with Determinantal Point Processes](https://mlsanity.com/paper/1907.04967/) (2019)
- [TPNet: Trajectory Proposal Network for Motion Prediction](https://mlsanity.com/paper/2004.12255/) (2020)
- [What-If Motion Prediction for Autonomous Driving](https://mlsanity.com/paper/2008.10587/) (2020)
- [Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs](https://mlsanity.com/paper/1805.05499/) (2018)
- [LaPred: Lane-Aware Prediction of Multi-Modal Future Trajectories of Dynamic Agents](https://mlsanity.com/paper/2104.00249/) (2021)
- [Learning to Predict Vehicle Trajectories with Model-based Planning](https://mlsanity.com/paper/2103.04027/) (2021)
- [Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data](https://mlsanity.com/paper/2001.03093/) (2020)

## Built on by (most cited first, 30 of 81)

- [Motion Transformer with Global Intention Localization and Local Movement Refinement](https://mlsanity.com/paper/2209.13508/) (2022)
- [MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion](https://mlsanity.com/paper/2306.03083/) (2023)
- [MotionLM: Multi-Agent Motion Forecasting as Language Modeling](https://mlsanity.com/paper/2309.16534/) (2023)
- [GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving](https://mlsanity.com/paper/2303.05760/) (2023)
- [MTR++: Multi-Agent Motion Prediction with Symmetric Scene Modeling and Guided Intention Querying](https://mlsanity.com/paper/2306.17770/) (2023)
- [CAT: Closed-loop Adversarial Training for Safe End-to-End Driving](https://mlsanity.com/paper/2310.12432/) (2023)
- [Traj-MAE: Masked Autoencoders for Trajectory Prediction](https://mlsanity.com/paper/2303.06697/) (2023)
- [FJMP: Factorized Joint Multi-Agent Motion Prediction over Learned Directed Acyclic Interaction Graphs](https://mlsanity.com/paper/2211.16197/) (2022)
- [JFP: Joint Future Prediction with Interactive Multi-Agent Modeling for Autonomous Driving](https://mlsanity.com/paper/2212.08710/) (2022)
- [Conditional Predictive Behavior Planning with Inverse Reinforcement Learning for Human-like Autonomous Driving](https://mlsanity.com/paper/2212.08787/) (2022)
- [Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding](https://mlsanity.com/paper/2310.12970/) (2023)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [Data-driven Traffic Simulation: A Comprehensive Review](https://mlsanity.com/paper/2310.15975/) (2023)
- [DriveGPT: Scaling Autoregressive Behavior Models for Driving](https://mlsanity.com/paper/2412.14415/) (2024)
- [GATraj: A Graph- and Attention-based Multi-Agent Trajectory Prediction Model](https://mlsanity.com/paper/2209.07857/) (2022)
- [CMP: Cooperative Motion Prediction with Multi-Agent Communication](https://mlsanity.com/paper/2403.17916/) (2024)
- [Critique of World Model](https://mlsanity.com/paper/2507.05169/) (2025)
- [Large Trajectory Models are Scalable Motion Predictors and Planners](https://mlsanity.com/paper/2310.19620/) (2023)
- [Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future](https://mlsanity.com/paper/2312.03408/) (2023)
- [Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generation](https://mlsanity.com/paper/2408.00766/) (2024)
- [Unsupervised Sampling Promoting for Stochastic Human Trajectory Prediction](https://mlsanity.com/paper/2304.04298/) (2023)
- [Joint Metrics Matter: A Better Standard for Trajectory Forecasting](https://mlsanity.com/paper/2305.06292/) (2023)
- [A Diffusion-Model of Joint Interactive Navigation](https://mlsanity.com/paper/2309.12508/) (2023)
- [IPCC-TP: Utilizing Incremental Pearson Correlation Coefficient for Joint Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2303.00575/) (2023)
- [AMP: Autoregressive Motion Prediction Revisited with Next Token Prediction for Autonomous Driving](https://mlsanity.com/paper/2403.13331/) (2024)
- [Context-Aware Timewise VAEs for Real-Time Vehicle Trajectory Prediction](https://mlsanity.com/paper/2302.10873/) (2023)
- [CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models](https://mlsanity.com/paper/2502.15119/) (2025)
- [Dream to Drive with Predictive Individual World Model](https://mlsanity.com/paper/2501.16733/) (2025)
- [Editing Driver Character: Socially-Controllable Behavior Generation for Interactive Traffic Simulation](https://mlsanity.com/paper/2303.13830/) (2023)
- [Domain Knowledge Driven Pseudo Labels for Interpretable Goal-Conditioned Interactive Trajectory Prediction](https://mlsanity.com/paper/2203.15112/) (2022)

## Abstract (arXiv)

Predicting future motions of road participants is an important task for driving autonomously in urban scenes. Existing models excel at predicting marginal trajectories for single agents, yet it remains an open question to jointly predict scene compliant trajectories over multiple agents. The challenge is due to exponentially increasing prediction space as a function of the number of agents. In this work, we exploit the underlying relations between interacting agents and decouple the joint prediction problem into marginal prediction problems. Our proposed approach M2I first classifies interacting agents as pairs of influencers and reactors, and then leverages a marginal prediction model and a conditional prediction model to predict trajectories for the influencers and reactors, respectively. The predictions from interacting agents are combined and selected according to their joint likelihoods. Experiments show that our simple but effective approach achieves state-of-the-art performance on the Waymo Open Motion Dataset interactive prediction benchmark.