# Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Dataset

arXiv: https://arxiv.org/abs/2506.22554 · HTML: https://mlsanity.com/paper/2506.22554/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jun 2025.

## Built on by (most cited first, 30 of 40)

- [GaP: A Graph-as-Policy Multi-Agent Self-Learning Harness For Variational Automation Tasks](https://mlsanity.com/paper/2607.05369/) (2026)
- [LPM 1.0: Video-based Character Performance Model](https://mlsanity.com/paper/2604.07823/) (2026)
- [Towards Interactive Intelligence for Digital Humans](https://mlsanity.com/paper/2512.13674/) (2025)
- [DyaDiT: A Multi-Modal Diffusion Transformer for Socially Favorable Dyadic Gesture Generation](https://mlsanity.com/paper/2602.23165/) (2026)
- [ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Body](https://mlsanity.com/paper/2512.14234/) (2025)
- [Towards Seamless Interaction: Causal Turn-Level Modeling of Interactive 3D Conversational Head Dynamics](https://mlsanity.com/paper/2512.15340/) (2025)
- [Dynamic Personality Adaptation in Large Language Models via State Machines](https://mlsanity.com/paper/2602.22157/) (2026)
- [Integrating Facial Generation into Full-Duplex Spoken Dialogue Systems](https://mlsanity.com/paper/2606.21970/) (2026)
- [Reference-Based Prosody and Rhythm Evaluation for Spoken Dialogue Systems](https://mlsanity.com/paper/2606.31055/) (2026)
- [FactorPortrait: Controllable Portrait Animation via Disentangled Expression, Pose, and Viewpoint](https://mlsanity.com/paper/2512.11645/) (2025)
- [Beyond Monologue: Interactive Talking-Listening Avatar Generation with Conversational Audio Context-Aware Kernels](https://mlsanity.com/paper/2604.10367/) (2026)
- [Mutual Forcing: Dual-Mode Self-Evolution for Fast Autoregressive Audio-Video Character Generation](https://mlsanity.com/paper/2604.25819/) (2026)
- [TRACE: Temporal Relationship-Aware Conversational Entrainment Detection in Dyadic Speech](https://mlsanity.com/paper/2606.30543/) (2026)
- [TurnNat: Automatic Evaluation of Turn-Taking Naturalness in Dyadic Spoken Dialogue](https://mlsanity.com/paper/2607.01345/) (2026)
- [How to Build Digital Humans? From Priors to Photorealistic Avatars](https://mlsanity.com/paper/2607.04341/) (2026)
- [InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generation](https://mlsanity.com/paper/2607.10287/) (2026)
- [ECHO: Towards Emotionally Appropriate and Contextually Aware Interactive Head Generation](https://mlsanity.com/paper/2603.17427/) (2026)
- [Personalizing Causal Audio-Driven Facial Motion via Dynamic Multi-modal Retrieval](https://mlsanity.com/paper/2604.23692/) (2026)
- [Recognizing Co-Speech Gestures in-the-Wild](https://mlsanity.com/paper/2605.31589/) (2026)
- [X-Stream: Exploring MLLMs as Multiplexers for Multi-Stream Understanding](https://mlsanity.com/paper/2606.02482/) (2026)
- [Real-time body pose non-verbal communication with a consistency-based reliability measure](https://mlsanity.com/paper/2606.09390/) (2026)
- [FacePlex: Toward Natural Full-Duplex Conversational Avatars](https://mlsanity.com/paper/2606.30145/) (2026)
- [COSI-Lab: Conference Living Lab for Modeling Multi-Perspective Multimodal Social Intention](https://mlsanity.com/paper/2607.28649/) (2026)
- [The GENEA Challenge 2026: A Large-Scale Disentangled Evaluation of Speech-Driven Gesture Generation on the Seamless Interaction Dataset](https://mlsanity.com/paper/2608.10839/) (2026)
- [Human-Centric Intelligence in the Era of Foundation Models: A Survey](https://mlsanity.com/paper/2608.18184/) (2026)
- [EditaLive! Unified Character Video Editing for Live Streaming](https://mlsanity.com/paper/2608.27123/) (2026)
- [When Text Misleads: Inconsistent-Aware Reasoning for Audio-Grounded Dialogue](https://mlsanity.com/paper/2608.27176/) (2026)
- [Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue](https://mlsanity.com/paper/2609.04250/) (2026)
- [Who Are They to Each Other? Multi-Agent Reasoning for Speaker Relationship Inference](https://mlsanity.com/paper/2609.09628/) (2026)
- [VoiceTrace: A Benchmark and Retrieval Framework for Who-Said-What Speech Retrieval](https://mlsanity.com/paper/2609.18521/) (2026)

## Abstract (arXiv)

Human communication involves a complex interplay of verbal and nonverbal signals, essential for conveying meaning and achieving interpersonal goals. To develop socially intelligent AI technologies, it is crucial to develop models that can both comprehend and generate dyadic behavioral dynamics. To this end, we introduce the Seamless Interaction Dataset, a large-scale collection of over 4,000 hours of face-to-face interaction footage from over 4,000 participants in diverse contexts. This dataset enables the development of AI technologies that understand dyadic embodied dynamics, unlocking breakthroughs in virtual agents, telepresence experiences, and multimodal content analysis tools. We also develop a suite of models that utilize the dataset to generate dyadic motion gestures and facial expressions aligned with human speech. These models can take as input both the speech and visual behavior of their interlocutors. We present a variant with speech from an LLM model and integrations with 2D and 3D rendering methods, bringing us closer to interactive virtual agents. Additionally, we describe controllable variants of our motion models that can adapt emotional responses and expressivity levels, as well as generating more semantically-relevant gestures. Finally, we discuss methods for assessing the quality of these dyadic motion models, which are demonstrating the potential for more intuitive and responsive human-AI interactions.