# An Open-source Benchmark of Deep Learning Models for Audio-visual Apparent and Self-reported Personality Recognition

arXiv: https://arxiv.org/abs/2210.09138 · HTML: https://mlsanity.com/paper/2210.09138/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 88 references that are in the map.
- First posted on arXiv 17 Oct 2022.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [An overview of gradient descent optimization algorithms](https://mlsanity.com/paper/1609.04747/) (2016)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [Temporal Pyramid Network for Action Recognition](https://mlsanity.com/paper/2004.03548/) (2020)
- [Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Dataset](https://mlsanity.com/paper/2012.14259/) (2020)
- [Dyadformer: A Multi-modal Transformer for Long-Range Modeling of Dyadic Interactions](https://mlsanity.com/paper/2109.09487/) (2021)
- [Deep Impression: Audiovisual Deep Residual Networks for Multimodal Apparent Personality Trait Recognition](https://mlsanity.com/paper/1609.05119/) (2016)
- [PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship](https://mlsanity.com/paper/1811.08657/) (2018)
- [First Impressions: A Survey on Vision-Based Apparent Personality Trait Analysis](https://mlsanity.com/paper/1804.08046/) (2018)
- [Bi-modal First Impressions Recognition using Temporally Ordered Deep Audio and Stochastic Visual Features](https://mlsanity.com/paper/1610.10048/) (2016)
- [Multimodal Video-based Apparent Personality Recognition Using Long Short-Term Memory and Convolutional Neural Networks](https://mlsanity.com/paper/1911.00381/) (2019)
- [Learning Graph Representation of Person-specific Cognitive Processes from Audio-visual Behaviours for Automatic Personality Recognition](https://mlsanity.com/paper/2110.13570/) (2021)
- [Modern Physiognomy: An Investigation on Predicting Personality Traits and Intelligence from the Human Face](https://mlsanity.com/paper/1604.07499/) (2016)
- [Socially-Aware Conference Participant Recommendation with Personality Traits](https://mlsanity.com/paper/2008.04653/) (2020)

## Built on by (most cited first, 12 of 12)

- [Pose as a Modality: A Psychology-Inspired Network for Personality Recognition with a New Multimodal Dataset](https://mlsanity.com/paper/2503.12912/) (2025)
- [Traits Run Deep: Enhancing Personality Assessment via Psychology-Guided LLM Representations and Multimodal Apparent Behaviors](https://mlsanity.com/paper/2507.22367/) (2025)
- [LLM-based Multimodal Personality Recognition via Facial Action Unit-Text Semantic Fusion](https://mlsanity.com/paper/2606.29900/) (2026)
- [PhoenixNest-Video: Evidence-Grounded Multimodal Agent Framework for Automated Video Interview Assessment](https://mlsanity.com/paper/2609.02231/) (2026)
- [Learning Person-specific Network Representation for Apparent Personality Traits Recognition](https://mlsanity.com/paper/2303.01236/) (2023)
- [Multi-modal expressive personality recognition in data non-ideal audiovisual based on multi-scale feature enhancement and modal augment](https://mlsanity.com/paper/2503.06108/) (2025)
- [CloSE: A Geometric Shape-Agnostic Cloth State Representation](https://mlsanity.com/paper/2504.05033/) (2025)
- [Learning Personalised Human Internal Cognition from External Expressive Behaviours for Real Personality Recognition](https://mlsanity.com/paper/2508.00205/) (2025)
- [Enhancing Personality Recognition by Comparing the Predictive Power of Traits, Facets, and Nuances](https://mlsanity.com/paper/2602.05650/) (2026)
- [Debiased Multimodal Personality Understanding through Dual Causal Intervention](https://mlsanity.com/paper/2605.06371/) (2026)
- [Traits Run Deeper: Trait-Specific Asymmetric Fusion for Multimodal Personality Assessment](https://mlsanity.com/paper/2606.11269/) (2026)
- [Frozen Multimodal Embeddings for AI-Assisted Interview Assessment of Personality and Cognitive Ability](https://mlsanity.com/paper/2606.11930/) (2026)

## Abstract (arXiv)

Personality determines a wide variety of human daily and working behaviours, and is crucial for understanding human internal and external states. In recent years, a large number of automatic personality computing approaches have been developed to predict either the apparent personality or self-reported personality of the subject based on non-verbal audio-visual behaviours. However, the majority of them suffer from complex and dataset-specific pre-processing steps and model training tricks. In the absence of a standardized benchmark with consistent experimental settings, it is not only impossible to fairly compare the real performances of these personality computing models but also makes them difficult to be reproduced. In this paper, we present the first reproducible audio-visual benchmarking framework to provide a fair and consistent evaluation of eight existing personality computing models (e.g., audio, visual and audio-visual) and seven standard deep learning models on both self-reported and apparent personality recognition tasks. Building upon a set of benchmarked models, we also investigate the impact of two previously-used long-term modelling strategies for summarising short-term/frame-level predictions on personality computing results. The results conclude: (i) apparent personality traits, inferred from facial behaviours by most benchmarked deep learning models, show more reliability than self-reported ones; (ii) visual models frequently achieved superior performances than audio models on personality recognition; (iii) non-verbal behaviours contribute differently in predicting different personality traits; and (iv) our reproduced personality computing models generally achieved worse performances than their original reported results. Our benchmark is publicly available at urlhttps://github.com/liaorongfan/DeepPersonality.