The Truth and Nothing but the Truth: Multimodal Analysis for Deception Detection
Not on the map. Why?
Abstract (arXiv)
We propose a data-driven method for automatic deception detection in real-life trial data using visual and verbal cues. Using OpenFace with facial action unit recognition, we analyze the movement of facial features of the witness when posed with questions and the acoustic patterns using OpenSmile. We then perform a lexical analysis on the spoken words, emphasizing the use of pauses and utterance breaks, feeding that to a Support Vector Machine to test deceit or truth prediction. We then try out a method to incorporate utterance-based fusion of visual and lexical analysis, using string based matching.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 4; 2022: 1; 2023: 1; 2024: 0; 2025: 0; 2026: 1.
7 papers.
- 2021Unsupervised Audio-Visual Subspace Alignment for High-Stakes Deception Detectioncited by 3
- 2022Can lies be faked? Comparing low-stakes and high-stakes deception video datasets from a Machine Learning perspectivecited by 2
- 2021Constructing Robust Emotional State-based Feature with a Novel Voting Scheme for Multi-modal Deception Detection in Videoscited by 1
- 2021Machine Learning-based Lie Detector applied to a Novel Annotated Game Datasetcited by 1
- 2021Affect-Aware Deep Belief Network Representations for Multimodal Unsupervised Deception Detectioncited by 1
- 2026When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platformscited by 1
- 2023Voting-based Multimodal Automatic Deception Detectioncited by 0
- 2026When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms18 Apr 2026
- 2023Voting-based Multimodal Automatic Deception Detection30 Jun 2023
- 2022Can lies be faked? Comparing low-stakes and high-stakes deception video datasets from a Machine Learning perspective23 Nov 2022
- 2021Affect-Aware Deep Belief Network Representations for Multimodal Unsupervised Deception Detection17 Aug 2021
- 2021Machine Learning-based Lie Detector applied to a Novel Annotated Game Dataset26 Apr 2021
- 2021Constructing Robust Emotional State-based Feature with a Novel Voting Scheme for Multi-modal Deception Detection in Videos16 Apr 2021
- 2021Unsupervised Audio-Visual Subspace Alignment for High-Stakes Deception Detection6 Feb 2021