# Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioning

arXiv: https://arxiv.org/abs/2404.05218 · HTML: https://mlsanity.com/paper/2404.05218/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 71 of its 91 references that are in the map.
- First posted on arXiv 8 Apr 2024.

## Builds on (most cited first, 30 of 71)

- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion](https://mlsanity.com/paper/2306.03083/) (2023)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [Long-term Human Motion Prediction with Scene Context](https://mlsanity.com/paper/2007.03672/) (2020)
- [From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting](https://mlsanity.com/paper/2012.01526/) (2020)
- [Putting People in their Place: Monocular Regression of 3D People in Depth](https://mlsanity.com/paper/2112.08274/) (2021)
- [Leapfrog Diffusion Model for Stochastic Trajectory Prediction](https://mlsanity.com/paper/2303.10895/) (2023)
- [DiffPose: Toward More Reliable 3D Pose Estimation](https://mlsanity.com/paper/2211.16940/) (2022)
- [PoseFormerV2: Exploring Frequency Domain for Efficient and Robust 3D Human Pose Estimation](https://mlsanity.com/paper/2303.17472/) (2023)
- [Remember Intentions: Retrospective-Memory-based Trajectory Prediction](https://mlsanity.com/paper/2203.11474/) (2022)
- [BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction](https://mlsanity.com/paper/2211.14304/) (2022)
- [Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregation](https://mlsanity.com/paper/2303.11579/) (2023)
- [HumanMAC: Masked Motion Completion for Human Motion Prediction](https://mlsanity.com/paper/2302.03665/) (2023)
- [Multi-Person Extreme Motion Prediction](https://mlsanity.com/paper/2105.08825/) (2021)
- [Generating Smooth Pose Sequences for Diverse Human Motion Prediction](https://mlsanity.com/paper/2108.08422/) (2021)
- [Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction](https://mlsanity.com/paper/2203.16051/) (2022)
- [DiffPose: Multi-hypothesis Human Pose Estimation using Diffusion models](https://mlsanity.com/paper/2211.16487/) (2022)
- [Multi-Person 3D Motion Prediction with Multi-Range Transformers](https://mlsanity.com/paper/2111.12073/) (2021)
- [MUSE-VAE: Multi-Scale VAE for Environment-Aware Long Term Trajectory Prediction](https://mlsanity.com/paper/2201.07189/) (2022)
- [Action-Agnostic Human Pose Forecasting](https://mlsanity.com/paper/1810.09676/) (2018)
- [TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild](https://mlsanity.com/paper/2104.04029/) (2021)
- [Socially and Contextually Aware Human Motion and Pose Forecasting](https://mlsanity.com/paper/2007.06843/) (2020)
- [GANet: Goal Area Network for Motion Forecasting](https://mlsanity.com/paper/2209.09723/) (2022)
- [Spatio-Temporal Gating-Adjacency GCN for Human Motion Prediction](https://mlsanity.com/paper/2203.01474/) (2022)
- [Leveraging Future Relationship Reasoning for Vehicle Trajectory Prediction](https://mlsanity.com/paper/2305.14715/) (2023)
- [Global-to-Local Modeling for Video-based 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2303.14747/) (2023)
- [Uncovering the Missing Pattern: Unified Framework Towards Trajectory Imputation and Prediction](https://mlsanity.com/paper/2303.16005/) (2023)
- [GLA-GCN: Global-local Adaptive Graph Convolutional Network for 3D Human Pose Estimation from Monocular Video](https://mlsanity.com/paper/2307.05853/) (2023)

## Built on by (most cited first, 30 of 30)

- [Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation](https://mlsanity.com/paper/2502.20370/) (2025)
- [MM-OR: A Large Multimodal Operating Room Dataset for Semantic Understanding of High-Intensity Surgical Environments](https://mlsanity.com/paper/2503.02579/) (2025)
- [Text2Interact: High-Fidelity and Diverse Text-to-Two-Person Interaction Generation](https://mlsanity.com/paper/2510.06504/) (2025)
- [Multi-modal Knowledge Distillation-based Human Trajectory Forecasting](https://mlsanity.com/paper/2503.22201/) (2025)
- [How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions](https://mlsanity.com/paper/2504.12284/) (2025)
- [Continuous Locomotive Crowd Behavior Generation](https://mlsanity.com/paper/2504.04756/) (2025)
- [Frequency-Guided Diffusion Model with Perturbation Training for Skeleton-Based Video Anomaly Detection](https://mlsanity.com/paper/2412.03044/) (2024)
- [Generative Active Learning for Long-tail Trajectory Prediction via Controllable Diffusion Model](https://mlsanity.com/paper/2507.22615/) (2025)
- [Interaction-Merged Motion Planning: Effectively Leveraging Diverse Motion Datasets for Robust Planning](https://mlsanity.com/paper/2507.04790/) (2025)
- [Waymo-3DSkelMo: A Multi-Agent 3D Skeletal Motion Dataset for Pedestrian Interaction Modeling in Autonomous Driving](https://mlsanity.com/paper/2508.09404/) (2025)
- [Large-Scale Multi-Character Interaction Synthesis](https://mlsanity.com/paper/2505.14087/) (2025)
- [Efficient Multi-Person Motion Prediction by Lightweight Spatial and Temporal Interactions](https://mlsanity.com/paper/2507.09446/) (2025)
- [Modeling 3D Pedestrian-Vehicle Interactions for Vehicle-Conditioned Pose Forecasting](https://mlsanity.com/paper/2602.08962/) (2026)
- [PolySLGen: Online Multimodal Speaking-Listening Reaction Generation in Polyadic Interaction](https://mlsanity.com/paper/2604.08125/) (2026)
- [Massively Multi-Person 3D Human Motion Forecasting with Scene Context](https://mlsanity.com/paper/2409.12189/) (2024)
- [HUMOF: Human Motion Forecasting in Interactive Social Scenes](https://mlsanity.com/paper/2506.03753/) (2025)
- [SEPose: A Synthetic Event-based Human Pose Estimation Dataset for Pedestrian Monitoring](https://mlsanity.com/paper/2507.11910/) (2025)
- [E-React: Towards Emotionally Controlled Synthesis of Human Reactions](https://mlsanity.com/paper/2508.06093/) (2025)
- [PV-WM: A Heterogeneous Micro-Macro World Model for Articulated Pedestrian-Vehicle Co-Rollout](https://mlsanity.com/paper/2609.07328/) (2026)
- [Ego-Human Motion Prediction with 3D-Aware LLM](https://mlsanity.com/paper/2607.07001/) (2026)
- [Scriboora: Rethinking Human Pose Forecasting](https://mlsanity.com/paper/2511.15565/) (2025)
- [Spatiotemporal-Untrammelled Mixture of Experts for Multi-Person Motion Prediction](https://mlsanity.com/paper/2512.21707/) (2025)
- [SimpliHuMoN: Simplifying Human Motion Prediction](https://mlsanity.com/paper/2603.04399/) (2026)
- [B'ezier Degradation Modeling for LiDAR-based Human Motion Capture](https://mlsanity.com/paper/2605.19620/) (2026)
- [Diverse Yet Consistent: Context-Guided Diffusion with Energy-Based Joint Refinement for Multi-Agent Motion Prediction](https://mlsanity.com/paper/2605.22017/) (2026)
- [Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos](https://mlsanity.com/paper/2606.18824/) (2026)
- [Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Prediction](https://mlsanity.com/paper/2608.03379/) (2026)
- [Multi-Person Human Motion Forecasting in Complex Scenes](https://mlsanity.com/paper/2608.27039/) (2026)
- [From Where to How: Continuous 4D Interaction Forecasting from Egocentric Video](https://mlsanity.com/paper/2609.08636/) (2026)
- [Adaptive Interaction Network for Human Motion Prediction During Human-Robot Collaboration](https://mlsanity.com/paper/2610.16624/) (2026)

## Abstract (arXiv)

Human pose forecasting garners attention for its diverse applications. However, challenges in modeling the multi-modal nature of human motion and intricate interactions among agents persist, particularly with longer timescales and more agents. In this paper, we propose an interaction-aware trajectory-conditioned long-term multi-agent human pose forecasting model, utilizing a coarse-to-fine prediction approach: multi-modal global trajectories are initially forecasted, followed by respective local pose forecasts conditioned on each mode. In doing so, our Trajectory2Pose model introduces a graph-based agent-wise interaction module for a reciprocal forecast of local motion-conditioned global trajectory and trajectory-conditioned local pose. Our model effectively handles the multi-modality of human motion and the complexity of long-term multi-agent interactions, improving performance in complex environments. Furthermore, we address the lack of long-term (6s+) multi-agent (5+) datasets by constructing a new dataset from real-world images and 2D annotations, enabling a comprehensive evaluation of our proposed model. State-of-the-art prediction performance on both complex and simpler datasets confirms the generalized effectiveness of our method. The code is available at https://github.com/Jaewoo97/T2P.