# Multimodal Video-based Apparent Personality Recognition Using Long Short-Term Memory and Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1911.00381 · HTML: https://mlsanity.com/paper/1911.00381/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2019.

## Built on by (most cited first, 7 of 7)

- [An Open-source Benchmark of Deep Learning Models for Audio-visual Apparent and Self-reported Personality Recognition](https://mlsanity.com/paper/2210.09138/) (2022)
- [Multimodal Personality Recognition using Cross-Attention Transformer and Behaviour Encoding](https://mlsanity.com/paper/2112.12180/) (2021)
- [Text-based automatic personality prediction: A bibliographic review](https://mlsanity.com/paper/2110.01186/) (2021)
- [Multimodal Vision Transformers with Forced Attention for Behavior Analysis](https://mlsanity.com/paper/2212.03968/) (2022)
- [Inferred vs traditional personality assessment: are we predicting the same thing?](https://mlsanity.com/paper/2103.09632/) (2021)
- [Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines](https://mlsanity.com/paper/2304.02313/) (2023)
- [Twenty Years of Personality Computing: Threats, Challenges and Future Directions](https://mlsanity.com/paper/2503.02082/) (2025)

## Abstract (arXiv)

Personality computing and affective computing, where the recognition of personality traits is essential, have gained increasing interest and attention in many research areas recently. We propose a novel approach to recognize the Big Five personality traits of people from videos. Personality and emotion affect the speaking style, facial expressions, body movements, and linguistic factors in social contexts, and they are affected by environmental elements. We develop a multimodal system to recognize apparent personality based on various modalities such as the face, environment, audio, and transcription features. We use modality-specific neural networks that learn to recognize the traits independently and we obtain a final prediction of apparent personality with a feature-level fusion of these networks. We employ pre-trained deep convolutional neural networks such as ResNet and VGGish networks to extract high-level features and Long Short-Term Memory networks to integrate temporal information. We train the large model consisting of modality-specific subnetworks using a two-stage training process. We first train the subnetworks separately and then fine-tune the overall model using these trained networks. We evaluate the proposed method using ChaLearn First Impressions V2 challenge dataset. Our approach obtains the best overall "mean accuracy" score, averaged over five personality traits, compared to the state-of-the-art.