# Deception Detection in Videos

arXiv: https://arxiv.org/abs/1712.04415 · HTML: https://mlsanity.com/paper/1712.04415/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2017.

## Built on by (most cited first, 20 of 20)

- [Audio-Visual Deception Detection: DOLOS Dataset and Parameter-Efficient Crossmodal Learning](https://mlsanity.com/paper/2303.12745/) (2023)
- [Benchmarking Cross-Domain Audio-Visual Deception Detection](https://mlsanity.com/paper/2405.06995/) (2024)
- [SVC 2025: the First Multimodal Deception Detection Challenge](https://mlsanity.com/paper/2508.04129/) (2025)
- [Unsupervised Audio-Visual Subspace Alignment for High-Stakes Deception Detection](https://mlsanity.com/paper/2102.03673/) (2021)
- [Deception Detection in Videos using the Facial Action Coding System](https://mlsanity.com/paper/2105.13659/) (2021)
- [Modeling Non-Cooperative Dialogue: Theoretical and Empirical Insights](https://mlsanity.com/paper/2207.07255/) (2022)
- [Explainable Verbal Deception Detection using Transformers](https://mlsanity.com/paper/2210.03080/) (2022)
- [Can lies be faked? Comparing low-stakes and high-stakes deception video datasets from a Machine Learning perspective](https://mlsanity.com/paper/2211.13035/) (2022)
- [Constructing Robust Emotional State-based Feature with a Novel Voting Scheme for Multi-modal Deception Detection in Videos](https://mlsanity.com/paper/2104.08373/) (2021)
- [Machine Learning-based Lie Detector applied to a Novel Annotated Game Dataset](https://mlsanity.com/paper/2104.12345/) (2021)
- [Affect-Aware Deep Belief Network Representations for Multimodal Unsupervised Deception Detection](https://mlsanity.com/paper/2108.07897/) (2021)
- [Flexible-modal Deception Detection with Audio-Visual Adapter](https://mlsanity.com/paper/2302.05727/) (2023)
- [DeceptionX: From Multimodal Evidence to Explainable Deception Detection](https://mlsanity.com/paper/2606.11385/) (2026)
- [A Review of Web Infodemic Analysis and Detection Trends across Multi-modalities using Deep Neural Networks](https://mlsanity.com/paper/2112.00803/) (2021)
- [Automated Deception Detection from Videos: Using End-to-End Learning Based High-Level Features and Classification Approaches](https://mlsanity.com/paper/2307.06625/) (2023)
- [LoRA-like Calibration for Multimodal Deception Detection using ATSFace Data](https://mlsanity.com/paper/2309.01383/) (2023)
- [DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning](https://mlsanity.com/paper/2603.23916/) (2026)
- [SVC 2026: the Second Multimodal Deception Detection Challenge and the First Domain Generalized Remote Physiological Measurement Challenge](https://mlsanity.com/paper/2604.05748/) (2026)
- [Artificial Intelligence can Recognize Whether a Job Applicant is Selling and/or Lying According to Facial Expressions and Head Movements Much More Correctly Than Human Interviewers](https://mlsanity.com/paper/2605.17461/) (2026)
- [Modality vs. Morphology: A Framework for Time Series Classification for Biological Signals](https://mlsanity.com/paper/2605.18483/) (2026)

## Abstract (arXiv)

We present a system for covert automated deception detection in real-life courtroom trial videos. We study the importance of different modalities like vision, audio and text for this task. On the vision side, our system uses classifiers trained on low level video features which predict human micro-expressions. We show that predictions of high-level micro-expressions can be used as features for deception prediction. Surprisingly, IDT (Improved Dense Trajectory) features which have been widely used for action recognition, are also very good at predicting deception in videos. We fuse the score of classifiers trained on IDT features and high-level micro-expressions to improve performance. MFCC (Mel-frequency Cepstral Coefficients) features from the audio domain also provide a significant boost in performance, while information from transcripts is not very beneficial for our system. Using various classifiers, our automated system obtains an AUC of 0.877 (10-fold cross-validation) when evaluated on subjects which were not part of the training set. Even though state-of-the-art methods use human annotations of micro-expressions for deception detection, our fully automated approach outperforms them by 5%. When combined with human annotations of micro-expressions, our AUC improves to 0.922. We also present results of a user-study to analyze how well do average humans perform on this task, what modalities they use for deception detection and how they perform if only one modality is accessible. Our project page can be found at urlhttps://doubaibai.github.io/DARE/.