mlsanityData to 7 Oct 2026

Towards Social Artificial Intelligence: Nonverbal Social Signal Prediction in A Triadic Interaction

arXiv 1906.04158 ↗First posted 10 Jun 2019cs.CV cs.AIcited by 48

Not on the map. Why?

Abstract (arXiv)

We present a new research task and a dataset to understand human social interactions via computational methods, to ultimately endow machines with the ability to encode and decode a broad channel of social signals humans use. This research direction is essential to make a machine that genuinely communicates with humans, which we call Social Artificial Intelligence. We first formulate the "social signal prediction" problem as a way to model the dynamics of social signals exchanged among interacting individuals in a data-driven way. We then present a new 3D motion capture dataset to explore this problem, where the broad spectrum of social signals (3D body, face, and hand motions) are captured in a triadic social interaction scenario. Baseline approaches to predict speaking status, social formation, and body gestures of interacting individuals are presented in the defined social prediction framework.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 8; 2023: 9; 2024: 10; 2025: 5; 2026: 5.

48 papers.

Sort:

By area:

  1. 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 136
  2. 2022Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motioncited by 125
  3. 2023Human Motion Generation: A Surveycited by 95
  4. 2024From Audio to Photoreal Embodiment: Synthesizing Humans in Conversationscited by 82
  5. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  6. 2021Responsive Listening Head Generation: A Benchmark Dataset and Baselinecited by 67
  7. 2023A Comprehensive Review of Data-Driven Co-Speech Gesture Generationcited by 66
  8. 2021iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysiscited by 63
  9. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 54
  10. 2022Interaction Transformer for Human Reaction Generationcited by 42
  11. 2023Can Language Models Learn to Listen?cited by 40
  12. 2022Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddingscited by 38
36 more
  1. 2023Affective Faces for Goal-Driven Dyadic Communicationcited by 34
  2. 2021Monocular 3D Reconstruction of Interacting Hands via Collision-Aware Factorized Refinementscited by 27
  3. 2024Towards Open Domain Text-Driven Synthesis of Multi-Person Motionscited by 27
  4. 2025A Survey on Human Interaction Motion Generationcited by 25
  5. 2022PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learningcited by 23
  6. 2021Learning Triadic Belief Dynamics in Nonverbal Communication from Videoscited by 17
  7. 2021HUMBI: A Large Multiview Dataset of Human Body Expressions and Benchmark Challengecited by 17
  8. 2023RenderIH: A Large-scale Synthetic Dataset for 3D Interacting Hand Pose Estimationcited by 16
  9. 2021Multimodal analysis of the predictability of hand-gesture propertiescited by 14
  10. 2023Interactive Conversational Head Generationcited by 14
  11. 2024Beyond Talking -- Generating Holistic 3D Human Dyadic Motion for Communicationcited by 14
  12. 2024Nonverbal Interaction Detectioncited by 14
  13. 2025SocialGen: Modeling Multi-Human Social Interaction with Language Modelscited by 14
  14. 2022Didn't see that coming: a survey on non-verbal social human behavior forecastingcited by 12
  15. 2025Towards Online Multi-Modal Social Interaction Understandingcited by 12
  16. 2023AgentAvatar: Disentangling Planning, Driving and Rendering for Photorealistic Avatar Agentscited by 11
  17. 2024HandOS: 3D Hand Reconstruction in One Stagecited by 9
  18. 2023Social Motion Prediction with Cognitive Hierarchiescited by 8
  19. 2024Identity-free Artificial Emotional Intelligence via Micro-Gesture Understandingcited by 8
  20. 2021Communicative Learning with Natural Gestures for Embodied Navigation Agents with Human-in-the-Scenecited by 7
  21. 2021Social Processes: Self-Supervised Meta-Learning over Conversational Groups for Forecasting Nonverbal Social Cuescited by 6
  22. 2022Snipper: A Spatiotemporal Transformer for Simultaneous Multi-Person 3D Pose Estimation Tracking and Forecasting on a Video Snippetcited by 6
  23. 2022Human-Robot Commensality: Bite Timing Prediction for Robot-Assisted Feeding in Groupscited by 5
  24. 2024LLM Gesticulator: Leveraging Large Language Models for Scalable and Controllable Co-Speech Gesture Synthesiscited by 5
  25. 2021DeepSocNav: Social Navigation by Imitating Human Behaviorscited by 3
  26. 2026PolySLGen: Online Multimodal Speaking-Listening Reaction Generation in Polyadic Interactioncited by 3
  27. 2025VM-BHINet:Vision Mamba Bimanual Hand Interaction Network for 3D Interacting Hand Mesh Recovery From a Single RGB Imagecited by 2
  28. 2026SARAH: Spatially Aware Real-time Agentic Humanscited by 2
  29. 2026MuPPet: Multi-person 2D-to-3D Pose Liftingcited by 2
  30. 2026iMiGUE-3K: A Large-Scale Benchmark for Micro-Gesture Analysis with Self-Supervised Learningcited by 2
  31. 2022No-audio speaking status detection in crowded settings via visual pose-based filtering and wearable accelerationcited by 1
  32. 2023TriSig: Assessing the statistical significance of triclusterscited by 1
  33. 2024REWIND Dataset: Privacy-preserving Speaking Status Segmentation from Multimodal Body Movement Signals in the Wildcited by 1
  34. 2024STMR: Spiral Transformer for Hand Mesh Reconstructioncited by 0
  35. 2025Learning to Generate Human-Human-Object Interactions from Textual Descriptionscited by 0
  36. 2026Human-Inspired Social Engagement Analysis via Interpretable Mutual Visual Attentioncited by 0