# Dynamic Emotion and Personality Profiling for Multimodal Deception Detection

arXiv: https://arxiv.org/abs/2604.17037 · HTML: https://mlsanity.com/paper/2604.17037/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 38 references that are in the map.
- First posted on arXiv 18 Apr 2026.

## Builds on (most cited first, 20 of 20)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [Is GPT-3 a Good Data Annotator?](https://mlsanity.com/paper/2212.10450/) (2022)
- [Want To Reduce Labeling Cost? GPT-3 Can Help](https://mlsanity.com/paper/2108.13487/) (2021)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://mlsanity.com/paper/2303.16854/) (2023)
- [On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection](https://mlsanity.com/paper/1811.07901/) (2018)
- [Deception Detection and Remote Physiological Monitoring: A Dataset and Baseline Experimental Results](https://mlsanity.com/paper/2106.06583/) (2021)
- [Multi-Granular Multimodal Clue Fusion for Meme Understanding](https://mlsanity.com/paper/2503.12560/) (2025)
- [Introducing Representations of Facial Affect in Automated Multimodal Deception Detection](https://mlsanity.com/paper/2008.13369/) (2020)
- [Zero-Shot Conversational Stance Detection: Dataset and Approaches](https://mlsanity.com/paper/2506.17693/) (2025)
- [Reverse Multi-Choice Dialogue Commonsense Inference with Graph-of-Thought](https://mlsanity.com/paper/2312.15291/) (2023)
- [MDPE: A Multimodal Deception Dataset with Personality and Emotional Characteristics](https://mlsanity.com/paper/2407.12274/) (2024)
- [Deception Detection in Group Video Conversations using Dynamic Interaction Networks](https://mlsanity.com/paper/2106.06163/) (2021)
- [Enhancing Hyperbole and Metaphor Detection with Their Bidirectional Dynamic Interaction and Emotion Knowledge](https://mlsanity.com/paper/2506.15504/) (2025)
- [Improving Human-Machine Cooperative Visual Search With Soft Highlighting](https://mlsanity.com/paper/1612.08117/) (2016)

## Built on by (most cited first, 1 of 1)

- [MMDS-Bench: Benchmarking Multimodal Large Language Models on Dynamic Stance in Social Media Interactions](https://mlsanity.com/paper/2608.30903/) (2026)

## Abstract (arXiv)

Deception detection is of great significance for ensuring information security and conducting public opinion analysis, with personality factors and emotion cues playing a critical role. However, existing methods lack sample-level dynamic annotations for emotions and personality.In this paper, we propose an innovative multi-model multi-prompt annotation scheme and a strict label quality evaluation standard, and establish a multimodal joint detection dataset DDEP for deception, emotion, and personality. Meanwhile, we propose Rel-DDEP, an adaptive reliability-weighted fusion framework. Our framework quantifies uncertainty by mapping modal features to a high-dimensional Gaussian distribution space. It then performs reliability-weighted fusion and incorporates an alignment module and a sorting constraint module to achieve joint detection of deception, emotion, and personality. Experimental results on the MDPE and DDEP datasets show that our Rel-DDEP significantly outperforms the existing state-of-the-art baseline models in three tasks. The F1 score of the deception detection increases by 2.53%, that of the emotion detection increases by 2.66%, and that of the personality detection increases by 9.30%. The experiments fully verify the necessity of annotating dynamic emotion and personality labels for each sample and the effectiveness of reliability-weighted fusion.