# Unsupervised Audio-Visual Subspace Alignment for High-Stakes Deception Detection

arXiv: https://arxiv.org/abs/2102.03673 · HTML: https://mlsanity.com/paper/2102.03673/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 28 references that are in the map.
- First posted on arXiv 6 Feb 2021.

## Builds on (most cited first, 6 of 6)

- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Deception Detection in Videos](https://mlsanity.com/paper/1712.04415/) (2017)
- [Face-Focused Cross-Stream Network for Deception Detection in Videos](https://mlsanity.com/paper/1812.04429/) (2018)
- [Introducing Representations of Facial Affect in Automated Multimodal Deception Detection](https://mlsanity.com/paper/2008.13369/) (2020)
- [The Truth and Nothing but the Truth: Multimodal Analysis for Deception Detection](https://mlsanity.com/paper/1903.04484/) (2019)
- [Automated Dyadic Data Recorder (ADDR) Framework and Analysis of Facial Cues in Deceptive Communication](https://mlsanity.com/paper/1709.02414/) (2017)

## Built on by (most cited first, 3 of 3)

- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [Benchmarking Cross-Domain Audio-Visual Deception Detection](https://mlsanity.com/paper/2405.06995/) (2024)
- [Affect-Aware Deep Belief Network Representations for Multimodal Unsupervised Deception Detection](https://mlsanity.com/paper/2108.07897/) (2021)

## Abstract (arXiv)

Automated systems that detect deception in high-stakes situations can enhance societal well-being across medical, social work, and legal domains. Existing models for detecting high-stakes deception in videos have been supervised, but labeled datasets to train models can rarely be collected for most real-world applications. To address this problem, we propose the first multimodal unsupervised transfer learning approach that detects real-world, high-stakes deception in videos without using high-stakes labels. Our subspace-alignment (SA) approach adapts audio-visual representations of deception in lab-controlled low-stakes scenarios to detect deception in real-world, high-stakes situations. Our best unsupervised SA models outperform models without SA, outperform human ability, and perform comparably to a number of existing supervised models. Our research demonstrates the potential for introducing subspace-based transfer learning to model high-stakes deception and other social behaviors in real-world contexts with a scarcity of labeled behavioral data.