# Deception Detection from Linguistic and Physiological Data Streams Using Bimodal Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2311.10944 · HTML: https://mlsanity.com/paper/2311.10944/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 65 references that are in the map.
- First posted on arXiv 18 Nov 2023.

## Builds on (most cited first, 13 of 13)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Can Language Understand Depth?](https://mlsanity.com/paper/2207.01077/) (2022)
- [Contextual Hourglass Network for Semantic Segmentation of High Resolution Aerial Imagery](https://mlsanity.com/paper/1810.12813/) (2018)
- [Improving Depth Gradient Continuity in Transformers: A Comparative Study on Monocular Depth Estimation with CNN](https://mlsanity.com/paper/2308.08333/) (2023)
- [Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance](https://mlsanity.com/paper/2311.01108/) (2023)
- [DSPoint: Dual-scale Point Cloud Recognition with High-frequency Fusion](https://mlsanity.com/paper/2111.10332/) (2021)
- [Deep Learning for Content-based Personalized Viewport Prediction of 360-Degree VR Videos](https://mlsanity.com/paper/2003.00429/) (2020)
- [Joint Coverage and Power Control in Highly Dynamic and Massive UAV Networks: An Aggregative Game-theoretic Learning Approach](https://mlsanity.com/paper/1907.08363/) (2019)
- [Encouraging Disentangled and Convex Representation with Controllable Interpolation Regularization](https://mlsanity.com/paper/2112.03163/) (2021)

## Built on by (most cited first, 30 of 56)

- [Exploring Diverse Methods in Visual Question Answering](https://mlsanity.com/paper/2404.13565/) (2024)
- [Regional Style and Color Transfer](https://mlsanity.com/paper/2404.13880/) (2024)
- [Harnessing Earnings Reports for Stock Predictions: A QLoRA-Enhanced LLM Approach](https://mlsanity.com/paper/2408.06634/) (2024)
- [A Comparative Study on Enhancing Prediction in Social Network Advertisement through Data Augmentation](https://mlsanity.com/paper/2404.13812/) (2024)
- [Enhance Image-to-Image Generation with LLaVA-generated Prompts](https://mlsanity.com/paper/2406.01956/) (2024)
- [Image anomaly detection and prediction scheme based on SSA optimized ResNet50-BiGRU model](https://mlsanity.com/paper/2406.13987/) (2024)
- [Cross-Task Multi-Branch Vision Transformer for Facial Expression and Mask Wearing Classification](https://mlsanity.com/paper/2404.14606/) (2024)
- [Evaluating Modern Approaches in 3D Scene Reconstruction: NeRF vs Gaussian-Based Methods](https://mlsanity.com/paper/2408.04268/) (2024)
- [Adaptive speed planning for Unmanned Vehicle Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2404.17379/) (2024)
- [Attention Mechanism and Context Modeling System for Text Mining Machine Translation](https://mlsanity.com/paper/2408.04216/) (2024)
- [Few-shot Name Entity Recognition on StackOverflow](https://mlsanity.com/paper/2404.09405/) (2024)
- [Efficiency optimization of large-scale language models based on deep learning in natural language processing tasks](https://mlsanity.com/paper/2405.11704/) (2024)
- [Research on reinforcement learning based warehouse robot navigation algorithm in complex warehouse layout](https://mlsanity.com/paper/2411.06128/) (2024)
- [NeuroBind: Towards Unified Multimodal Representations for Neural Signals](https://mlsanity.com/paper/2407.14020/) (2024)
- [Application of Multimodal Fusion Deep Learning Model in Disease Recognition](https://mlsanity.com/paper/2406.18546/) (2024)
- [Adaptive Fault Tolerance Mechanisms of Large Language Models in Cloud Computing Environments](https://mlsanity.com/paper/2503.12228/) (2025)
- [Mix of Experts Language Model for Named Entity Recognition](https://mlsanity.com/paper/2404.19192/) (2024)
- [Exploration of Attention Mechanism-Enhanced Deep Learning Models in the Mining of Medical Textual Data](https://mlsanity.com/paper/2406.00016/) (2024)
- [Enhancing Medical Imaging with GANs Synthesizing Realistic Images from Limited Data](https://mlsanity.com/paper/2406.18547/) (2024)
- [Research on Driver Facial Fatigue Detection Based on Yolov8 Model](https://mlsanity.com/paper/2406.18575/) (2024)
- [Exploration of Multi-Scale Image Fusion Systems in Intelligent Medical Image Analysis](https://mlsanity.com/paper/2406.18548/) (2024)
- [Advanced User Credit Risk Prediction Model using LightGBM, XGBoost and Tabnet with SMOTEENN](https://mlsanity.com/paper/2408.03497/) (2024)
- [Cloud-Based AI Systems: Leveraging Large Language Models for Intelligent Fault Detection and Autonomous Self-Healing](https://mlsanity.com/paper/2505.11743/) (2025)
- [Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challenge](https://mlsanity.com/paper/2405.16464/) (2024)
- [Exploring Behavior-Relevant and Disentangled Neural Dynamics with Generative Diffusion Models](https://mlsanity.com/paper/2410.09614/) (2024)
- [Data Augmentation Through Random Style Replacement](https://mlsanity.com/paper/2504.10563/) (2025)
- [Cross-Cloud Data Privacy Protection: Optimizing Collaborative Mechanisms of AI Systems by Integrating Federated Learning and LLMs](https://mlsanity.com/paper/2505.13292/) (2025)
- [AD-Aligning: Emulating Human-like Generalization for Cognitive Domain Adaptation in Deep Learning](https://mlsanity.com/paper/2405.09582/) (2024)
- [Integrating Medical Imaging and Clinical Reports Using Multimodal Deep Learning for Advanced Disease Analysis](https://mlsanity.com/paper/2405.17459/) (2024)
- [Application of Natural Language Processing in Financial Risk Detection](https://mlsanity.com/paper/2406.09765/) (2024)

## Abstract (arXiv)

Deception detection is gaining increasing interest due to ethical and security concerns. This paper explores the application of convolutional neural networks for the purpose of multimodal deception detection. We use a dataset built by interviewing 104 subjects about two topics, with one truthful and one falsified response from each subject about each topic. In particular, we make three main contributions. First, we extract linguistic and physiological features from this data to train and construct the neural network models. Second, we propose a fused convolutional neural network model using both modalities in order to achieve an improved overall performance. Third, we compare our new approach with earlier methods designed for multimodal deception detection. We find that our system outperforms regular classification methods; our results indicate the feasibility of using neural networks for deception detection even in the presence of limited amounts of data.