mlsanityData to 7 Oct 2026

InterAct: Capture and Modelling of Realistic, Expressive and Interactive Activities between Two Persons in Daily Scenarios

arXiv 2405.11690 ↗First posted 19 May 2024cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We address the problem of accurate capture and expressive modelling of interactive behaviors happening between two persons in daily scenarios. Different from previous works which either only consider one person or focus on conversational gestures, we propose to simultaneously model the activities of two persons, and target objective-driven, dynamic, and coherent interactions which often span long duration. To this end, we capture a new dataset dubbed InterAct, which is composed of 241 motion sequences where two persons perform a realistic scenario over the whole sequence. The audios, body motions, and facial expressions of both persons are all captured in our dataset. We also demonstrate the first diffusion model based approach that directly estimates the interactive motions between two persons from their audios alone. All the data and code will be available at: https://hku-cg.github.io/interact.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 28 of its 44 references that are in the map. First posted on arXiv 19 May 2024.

Builds on

28 of 44 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2022Human Motion Diffusion Modelcited by 1,062
  6. 2020AdapterHub: A Framework for Adapting Transformerscited by 441
  7. 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 257
  8. 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 208
  9. 2019Learning Individual Styles of Conversational Gesturecited by 190
  10. 2021MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglementcited by 189
  11. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 188
  12. 2023CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Priorcited by 183
16 more
  1. 2022Generating Holistic 3D Human Motion from Speechcited by 180
  2. 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 177
  3. 2023GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latentscited by 156
  4. 2023EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animationcited by 141
  5. 2022Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motioncited by 125
  6. 2023Emotional Speech-Driven Animation with Content-Emotion Disentanglementcited by 98
  7. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  8. 2023Hi4D: 4D Instance Segmentation of Close Human Interactioncited by 87
  9. 2022ZeroEGGS: Zero-shot Example-based Gesture Generation from Speechcited by 83
  10. 2024From Audio to Photoreal Embodiment: Synthesizing Humans in Conversationscited by 82
  11. 2023FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusioncited by 77
  12. 2023Can Language Models Learn to Listen?cited by 40
  13. 2020Body2Hands: Learning to Infer 3D Hands from Conversational Gesture Body Dynamicscited by 34
  14. 2023BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformercited by 21
  15. 2024DiffSpeaker: Speech-Driven 3D Facial Animation with Diffusion Transformercited by 18
  16. 2021Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animationcited by 15

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 6.

7 papers.

  1. 2024INFP: Audio-Driven Interactive Head Generation in Dyadic Conversationscited by 40
  2. 2025ARIG: Autoregressive Interactive Head Generation for Real-time Conversationscited by 16
  3. 2025SocialGen: Modeling Multi-Human Social Interaction with Language Modelscited by 14
  4. 2025MOSPA: Human Motion Generation Driven by Spatial Audiocited by 11
  5. 2025Social Agent: Mastering Dyadic Nonverbal Behavior Generation via Conversational LLM Agentscited by 8
  6. 2025DyStream: Streaming Dyadic Talking Heads Generation via Flow Matching-based Autoregressive Modelcited by 5
  7. 2025PhysiInter: Integrating Physical Mapping for High-Fidelity Human Interaction Generationcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2025Multi-Person Interaction Generation from Two-Person Motion Priors
  2. 2024It Takes Two: Real-time Co-Speech Two-person's Interaction Generation via Reactive Auto-regressive Diffusion Model
  3. 2025InteracTalker: Prompt-Based Human-Object Interaction with Co-Speech Gesture Generation
  4. 2026COSMI: COmpositional Synthesis of Multi-object Interactions
  5. 2025Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Models
  6. 2026DyaDiT: A Multi-Modal Diffusion Transformer for Socially Favorable Dyadic Gesture Generation
  7. 2024in2IN: Leveraging individual Information to Generate Human INteractions
  8. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction