# Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowds

arXiv: https://arxiv.org/abs/2105.06496 · HTML: https://mlsanity.com/paper/2105.06496/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 95 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 26 of 26)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://mlsanity.com/paper/1905.08233/) (2019)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [The Creation and Detection of Deepfakes: A Survey](https://mlsanity.com/paper/2004.11138/) (2020)
- [DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection](https://mlsanity.com/paper/2001.00179/) (2020)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [The Deepfake Detection Challenge (DFDC) Preview Dataset](https://mlsanity.com/paper/1910.08854/) (2019)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [The DeepFake Detection Challenge (DFDC) Dataset](https://mlsanity.com/paper/2006.07397/) (2020)
- [GridMask Data Augmentation](https://mlsanity.com/paper/2001.04086/) (2020)
- [FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces](https://mlsanity.com/paper/1803.09179/) (2018)
- [Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues](https://mlsanity.com/paper/2003.06711/) (2020)
- [Detecting Deep-Fake Videos from Appearance and Behavior](https://mlsanity.com/paper/2004.14491/) (2020)
- [DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms](https://mlsanity.com/paper/2006.07634/) (2020)
- [DeepFake Detection: Current Challenges and Next Steps](https://mlsanity.com/paper/2003.09234/) (2020)
- [The Deepfake Detection Dilemma: A Multistakeholder Exploration of Adversarial Dynamics in Synthetic Media](https://mlsanity.com/paper/2102.06109/) (2021)
- [Human detection of machine manipulated media](https://mlsanity.com/paper/1907.05276/) (2019)
- [Could you become more credible by being White? Assessing Impact of Race on Credibility with Deepfakes](https://mlsanity.com/paper/2102.08054/) (2021)

## Built on by (most cited first, 30 of 58)

- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [Best Practices and Lessons Learned on Synthetic Data](https://mlsanity.com/paper/2404.07503/) (2024)
- [When combinations of humans and AI are useful: A systematic review and meta-analysis](https://mlsanity.com/paper/2405.06087/) (2024)
- [Art and the science of generative AI: A deeper dive](https://mlsanity.com/paper/2306.04141/) (2023)
- [Voice-Face Homogeneity Tells Deepfake](https://mlsanity.com/paper/2203.02195/) (2022)
- [FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models](https://mlsanity.com/paper/2404.13306/) (2024)
- [Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithm](https://mlsanity.com/paper/2207.02942/) (2022)
- [As Good As A Coin Toss: Human detection of AI-generated images, videos, audio, and audiovisual stimuli](https://mlsanity.com/paper/2403.16760/) (2024)
- [Learning When to Advise Human Decision Makers](https://mlsanity.com/paper/2209.13578/) (2022)
- [Human Detection of Political Speech Deepfakes across Transcripts, Audio, and Video](https://mlsanity.com/paper/2202.12883/) (2022)
- [Human Perception of Audio Deepfakes](https://mlsanity.com/paper/2107.09667/) (2021)
- [Testing Human Ability To Detect Deepfake Images of Human Faces](https://mlsanity.com/paper/2212.05056/) (2022)
- [Characterizing Photorealism and Artifacts in Diffusion Model-Generated Images](https://mlsanity.com/paper/2502.11989/) (2025)
- [People are poorly equipped to detect AI-powered voice clones](https://mlsanity.com/paper/2410.03791/) (2024)
- [Reinforced Disentanglement for Face Swapping without Skip Connection](https://mlsanity.com/paper/2307.07928/) (2023)
- [AI-enhanced Collective Intelligence](https://mlsanity.com/paper/2403.10433/) (2024)
- [A Test for Evaluating Performance in Human-Computer Systems](https://mlsanity.com/paper/2206.12390/) (2022)
- [Non-Consensual Synthetic Intimate Imagery: Prevalence, Attitudes, and Knowledge in 10 Countries](https://mlsanity.com/paper/2402.01721/) (2024)
- [GOTCHA: Real-Time Video Deepfake Detection via Challenge-Response](https://mlsanity.com/paper/2210.06186/) (2022)
- [On pitfalls (and advantages) of sophisticated large language models](https://mlsanity.com/paper/2303.17511/) (2023)
- [Unmasking Illusions: Understanding Human Perception of Audiovisual Deepfakes](https://mlsanity.com/paper/2405.04097/) (2024)
- [Synthetic Human Memories: AI-Edited Images and Videos Can Implant False Memories and Distort Recollection](https://mlsanity.com/paper/2409.08895/) (2024)
- [The DeepSpeak Dataset](https://mlsanity.com/paper/2408.05366/) (2024)
- [Explaining and Improving Information Complementarities in Multi-Agent Decision-making](https://mlsanity.com/paper/2502.06152/) (2025)
- [Identifying the Context Shift between Test Benchmarks and Production Data](https://mlsanity.com/paper/2207.01059/) (2022)
- [Trucks Don't Mean Trump: Diagnosing Human Error in Image Analysis](https://mlsanity.com/paper/2205.07333/) (2022)
- [Deceptive AI systems that give explanations are more convincing than honest AI systems and can amplify belief in misinformation](https://mlsanity.com/paper/2408.00024/) (2024)
- [Protecting President Zelenskyy against Deep Fakes](https://mlsanity.com/paper/2206.12043/) (2022)
- [Signals of Provenance: Practices & Challenges of Navigating Indicators in AI-Generated Media for Sighted and Blind Individuals](https://mlsanity.com/paper/2505.16057/) (2025)
- [All in One: Unifying Deepfake Detection, Tampering Localization, and Source Tracing with a Robust Landmark-Identity Watermark](https://mlsanity.com/paper/2602.23523/) (2026)

## Abstract (arXiv)

The recent emergence of machine-manipulated media raises an important societal question: how can we know if a video that we watch is real or fake? In two online studies with 15,016 participants, we present authentic videos and deepfakes and ask participants to identify which is which. We compare the performance of ordinary human observers against the leading computer vision deepfake detection model and find them similarly accurate while making different kinds of mistakes. Together, participants with access to the model's prediction are more accurate than either alone, but inaccurate model predictions often decrease participants' accuracy. To probe the relative strengths and weaknesses of humans and machines as detectors of deepfakes, we examine human and machine performance across video-level features, and we evaluate the impact of pre-registered randomized interventions on deepfake detection. We find that manipulations designed to disrupt visual processing of faces hinder human participants' performance while mostly not affecting the model's performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance.