mlsanityData to 7 Oct 2026

Comparison of Spatio-Temporal Models for Human Motion and Pose Forecasting in Face-to-Face Interaction Scenarios

arXiv 2203.03245 ↗First posted 7 Mar 2022cs.CV cs.ROcited by 8

Not on the map. Why?

Abstract (arXiv)

Human behavior forecasting during human-human interactions is of utmost importance to provide robotic or virtual agents with social intelligence. This problem is especially challenging for scenarios that are highly driven by interpersonal dynamics. In this work, we present the first systematic comparison of state-of-the-art approaches for behavior forecasting. To do so, we leverage whole-body annotations (face, body, and hands) from the very recently released UDIVA v0.5, which features face-to-face dyadic interactions. Our best attention-based approaches achieve state-of-the-art performance in UDIVA v0.5. We show that by autoregressively predicting the future with methods trained for the short-term future (<400ms), we outperform the baselines even for a considerably longer-term future (up to 2s). We also show that this finding holds when highly noisy annotations are used, which opens new horizons towards the use of weakly-supervised learning. Combined with large-scale datasets, this may help boost the advances in this field.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 35 of its 62 references that are in the map. First posted on arXiv 7 Mar 2022.

Builds on

35 of 62 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  6. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  7. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  8. 2020Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networkscited by 563
  9. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  10. 2017On human motion prediction using recurrent neural networkscited by 336
  11. 2020Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Predictioncited by 324
  12. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
23 more
  1. 2018Deep Appearance Models for Face Renderingcited by 164
  2. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 164
  3. 2018Convolutional Sequence to Sequence Model for Human Dynamicscited by 115
  4. 2019Context-aware Human Motion Predictioncited by 83
  5. 2017Learning Human Motion Models for Long-term Predictionscited by 67
  6. 2021Multi-Person 3D Motion Prediction with Multi-Range Transformerscited by 63
  7. 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
  8. 2020Socially and Contextually Aware Human Motion and Pose Forecastingcited by 51
  9. 2019Symbiotic Graph Neural Networks for 3D Skeleton-based Human Action Recognition and Motion Predictioncited by 50
  10. 2021Multi-level Motion Attention for Human Motion Predictioncited by 35
  11. 2020Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Datasetcited by 31
  12. 2019To React or not to React: End-to-End Visual Pose Forecasting for Personalized Avatar during Dyadic Conversationscited by 26
  13. 2019Looking Ahead: Anticipating Pedestrians Crossing with Future Frames Predictioncited by 25
  14. 2021Simple Baseline for Single Human Motion Forecastingcited by 18
  15. 2020Forecasting Characteristic 3D Poses of Human Actionscited by 16
  16. 2021TrajeVAE: Controllable Human Motion Generation from Trajectoriescited by 14
  17. 2022Didn't see that coming: a survey on non-verbal social human behavior forecastingcited by 12
  18. 2021Dyadic Human Motion Predictioncited by 9
  19. 2021Social Processes: Self-Supervised Meta-Learning over Conversational Groups for Forecasting Nonverbal Social Cuescited by 6
  20. 2021Fast User Adaptation for Human Motion Prediction in Physical Human-Robot Interactioncited by 6
  21. 2020DeepSSM: Deep State-Space Model for 3D Human Motion Predictioncited by 3
  22. 2019Towards More Realistic Human-Robot Conversation: A Seq2Seq-based Body Gesture Interaction Systemcited by 2
  23. 2021Forecasting Nonverbal Social Signals during Dyadic Interactions with Generative Adversarial Neural Networkscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 3; 2025: 3.

8 papers.

  1. 2022BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Predictioncited by 82
  2. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 77
  3. 2024in2IN: Leveraging individual Information to Generate Human INteractionscited by 29
  4. 2024REACT 2024: the Second Multiple Appropriate Facial Reaction Generation Challengecited by 16
  5. 2025Nonisotropic Gaussian Diffusion for Realistic 3D Human Motion Predictioncited by 12
  6. 2025From Sparse Signal to Smooth Motion: Real-Time Motion Generation with Rolling Prediction Modelscited by 11
  7. 2023REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challengecited by 5
  8. 2025Latent Behavior Diffusion for Sequential Reaction Generation in Dyadic Settingcited by 3