mlsanityData to 14 Oct 2026

Multimodal Personality Recognition using Cross-Attention Transformer and Behaviour Encoding

arXiv 2112.12180 ↗First posted 22 Dec 2021cs.CV cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Personality computing and affective computing have gained recent interest in many research areas. The datasets for the task generally have multiple modalities like video, audio, language and bio-signals. In this paper, we propose a flexible model for the task which exploits all available data. The task involves complex relations and to avoid using a large model for video processing specifically, we propose the use of behaviour encoding which boosts performance with minimal change to the model. Cross-attention using transformers has become popular in recent times and is utilised for fusion of different modalities. Since long term relations may exist, breaking the input into chunks is not desirable, thus the proposed model processes the entire input together. Our experiments show the importance of each of the above contributions

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 14 of its 34 references that are in the map. First posted on arXiv 22 Dec 2021.

Builds on

14 of 34 references.

References · 14

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  4. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  5. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
  6. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
  7. 2018LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Imagescited by 88
  8. 2021Multimodal End-to-End Sparse Model for Emotion Recognitioncited by 40
  9. 2020Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Datasetcited by 31
  10. 2016Deep Impression: Audiovisual Deep Residual Networks for Multimodal Apparent Personality Trait Recognitioncited by 15
  11. 2016Bi-modal First Impressions Recognition using Temporally Ordered Deep Audio and Stochastic Visual Featurescited by 13
  12. 2019Multimodal Video-based Apparent Personality Recognition Using Long Short-Term Memory and Convolutional Neural Networkscited by 7
2 more
  1. 2020Person Perception Biases Exposed: Revisiting the First Impressions Datasetcited by 5
  2. 2016Modern Physiognomy: An Investigation on Predicting Personality Traits and Intelligence from the Human Facecited by 3

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 0; 2024: 1; 2025: 2.

5 papers.

  1. 2022Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluationcited by 19
  2. 2022Multimodal Vision Transformers with Forced Attention for Behavior Analysiscited by 3
  3. 2025Graph-Driven Multimodal Feature Learning Framework for Apparent Personality Assessmentcited by 2
  4. 2024Loose Social-Interaction Recognition in Real-world Therapy Scenarioscited by 0
  5. 2025CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasetscited by 0