# Deception Detection in Dyadic Exchanges Using Multimodal Machine Learning: A Study on a Swedish Cohort

arXiv: https://arxiv.org/abs/2506.21429 · HTML: https://mlsanity.com/paper/2506.21429/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 50 references that are in the map.
- First posted on arXiv 26 Jun 2025.

## Builds on (most cited first, 5 of 5)

- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels](https://mlsanity.com/paper/1910.13051/) (2019)
- [MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records](https://mlsanity.com/paper/2102.02340/) (2021)
- [Automated Dyadic Data Recorder (ADDR) Framework and Analysis of Facial Cues in Deceptive Communication](https://mlsanity.com/paper/1709.02414/) (2017)
- [Can lies be faked? Comparing low-stakes and high-stakes deception video datasets from a Machine Learning perspective](https://mlsanity.com/paper/2211.13035/) (2022)

## Abstract (arXiv)

This study investigates the efficacy of using multimodal machine learning techniques to detect deception in dyadic interactions, focusing on the integration of data from both the deceiver and the deceived. We compare early and late fusion approaches, utilizing audio and video data - specifically, Action Units and gaze information - across all possible combinations of modalities and participants. Our dataset, newly collected from Swedish native speakers engaged in truth or lie scenarios on emotionally relevant topics, serves as the basis for our analysis. The results demonstrate that incorporating both speech and facial information yields superior performance compared to single-modality approaches. Moreover, including data from both participants significantly enhances deception detection accuracy, with the best performance (71%) achieved using a late fusion strategy applied to both modalities and participants. These findings align with psychological theories suggesting differential control of facial and vocal expressions during initial interactions. As the first study of its kind on a Scandinavian cohort, this research lays the groundwork for future investigations into dyadic interactions, particularly within psychotherapy settings.