# MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group Settings

arXiv: https://arxiv.org/abs/2309.10765 · HTML: https://mlsanity.com/paper/2309.10765/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 19 Sep 2023.

## Builds on (most cited first, 9 of 9)

- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Learning Video Representations from Large Language Models](https://mlsanity.com/paper/2212.04501/) (2022)
- [Detecting Low Rapport During Natural Interactions in Small Groups from Non-Verbal Behaviour](https://mlsanity.com/paper/1801.06055/) (2018)
- [Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluation](https://mlsanity.com/paper/2207.12817/) (2022)
- [Multichannel Attention Network for Analyzing Visual Behavior in Public Speaking](https://mlsanity.com/paper/1707.06830/) (2017)
- [Head Matters: Explainable Human-centered Trait Prediction from Head Motion Dynamics](https://mlsanity.com/paper/2112.08068/) (2021)
- [MultiMediate'23: Engagement Estimation and Bodily Behaviour Recognition in Social Interactions](https://mlsanity.com/paper/2308.08256/) (2023)
- [Explainable Human-centered Traits from Head Motion and Facial Expression Dynamics](https://mlsanity.com/paper/2302.09817/) (2023)

## Built on by (most cited first, 1 of 1)

- [MIP-GAF: A MLLM-annotated Benchmark for Most Important Person Localization and Group Context Understanding](https://mlsanity.com/paper/2409.06224/) (2024)

## Abstract (arXiv)

Bodily behavioral language is an important social cue, and its automated analysis helps in enhancing the understanding of artificial intelligence systems. Furthermore, behavioral language cues are essential for active engagement in social agent-based user interactions. Despite the progress made in computer vision for tasks like head and body pose estimation, there is still a need to explore the detection of finer behaviors such as gesturing, grooming, or fumbling. This paper proposes a multiview attention fusion method named MAGIC-TBR that combines features extracted from videos and their corresponding Discrete Cosine Transform coefficients via a transformer-based approach. The experiments are conducted on the BBSI dataset and the results demonstrate the effectiveness of the proposed feature fusion with multiview attention. The code is available at: https://github.com/surbhimadan92/MAGIC-TBR