mlsanityData to 7 Oct 2026

MultiMediate'23: Engagement Estimation and Bodily Behaviour Recognition in Social Interactions

arXiv 2308.08256 ↗First posted 16 Aug 2023cs.CV cs.HCcited by 3

Not on the map. Why?

Abstract (arXiv)

Automatic analysis of human behaviour is a fundamental prerequisite for the creation of machines that can effectively interact with- and support humans in social interactions. In MultiMediate'23, we address two key human social behaviour analysis tasks for the first time in a controlled challenge: engagement estimation and bodily behaviour recognition in social interactions. This paper describes the MultiMediate'23 challenge and presents novel sets of annotations for both tasks. For engagement estimation we collected novel annotations on the NOvice eXpert Interaction (NOXI) database. For bodily behaviour recognition, we annotated test recordings of the MPIIGroupInteraction corpus with the BBSI annotation scheme. In addition, we present baseline results for both challenge tasks.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 15 of its 62 references that are in the map. First posted on arXiv 16 Aug 2023.

Builds on

15 of 62 references.

References · 15

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  3. 2016SoundNet: Learning Sound Representations from Unlabeled Videocited by 244
  4. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  5. 2016AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challengecited by 68
  6. 2021iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysiscited by 63
  7. 2018Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviourcited by 24
  8. 2022Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluationcited by 19
  9. 2022MultiMediate '22: Backchannel Detection and Agreement Estimation in Group Interactionscited by 10
  10. 2020Towards Social & Engaging Peer Learning: Predicting Backchanneling and Disengagement in Childrencited by 7
  11. 2023Backchannel Detection and Agreement Estimation from Video with Transformer Networkscited by 7
  12. 2019Emergent Leadership Detection Across Datasetscited by 5
3 more
  1. 2020Modeling Engagement in Long-Term, In-Home Socially Assistive Robot Interventions for Children with Autism Spectrum Disorderscited by 4
  2. 2020Can I Trust You? A User Study of Robot Mediation of a Support Groupcited by 1
  3. 2020Developing an Effective and Automated Patient Engagement Estimator for Telehealth: A Machine Learning Approachcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 1.

3 papers.

  1. 2024Multilingual Dyadic Interaction Corpus NoXi+J: Toward Understanding Asian-European Non-verbal Cultural Characteristics and their Influences on Engagementcited by 5
  2. 2025Beyond Words: Multimodal LLM Knows When to Speakcited by 4
  3. 2023MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group Settingscited by 1