A Synchronized Audio-Visual Multi-View Capture System
Not on the map. Why?
Abstract (arXiv)
Multi-view capture systems have been an important tool in research for recording human motion under controlling conditions. Most existing systems are specified around video streams and provide little or no support for audio acquisition and rigorous audio-video alignment, despite both being essential for studying conversational interaction where timing at the level of turn-taking, overlap, and prosody matters. In this technical report, we describe an audio-visual multi-view capture system that addresses this gap by treating synchronized audio and synchronized video as first-class signals. The system combines a multi-camera pipeline with multi-channel microphone recording under a unified timing architecture and provides a practical workflow for calibration, acquisition, and quality control that supports repeatable recordings at scale. We quantify synchronization performance in deployment and show that the resulting recordings are temporally consistent enough to support fine-grained analysis and data-driven modeling of conversation behavior.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 13 of its 27 references that are in the map. First posted on arXiv 24 Mar 2026.
Builds on
13 of 27 references.
References · 13
- 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
- 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
- 2016Convolutional Pose Machinescited by 325
- 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
- 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
- 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 177
- 2023Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generationcited by 128
- 2021Learning Speech-driven 3D Conversational Gestures from Videocited by 98
- 2023StyleTalk: One-shot Talking Head Generation with Controllable Speaking Stylescited by 92
- 2020Audio- and Gaze-driven Facial Animation of Codec Avatarscited by 46
- 2022ConfLab: A Data Collection Concept, Dataset, and Benchmark for Machine Analysis of Free-Standing Social Interactions in the Wildcited by 9
- 2025Co3Gesture: Towards Coherent Concurrent Co-speech 3D Gesture Generation with Interactive Diffusioncited by 9
Built on by
Not cited yet.