# PolyGlotFake: A Novel Multilingual and Multimodal DeepFake Dataset

arXiv: https://arxiv.org/abs/2405.08838 · HTML: https://mlsanity.com/paper/2405.08838/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 14 May 2024.

## Builds on (most cited first, 30 of 30)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [Generalizing Face Forgery Detection with High-frequency Features](https://mlsanity.com/paper/2103.12376/) (2021)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasets](https://mlsanity.com/paper/2104.09494/) (2021)
- [FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset](https://mlsanity.com/paper/2108.05080/) (2021)
- [FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces](https://mlsanity.com/paper/1803.09179/) (2018)
- [M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection](https://mlsanity.com/paper/2104.09770/) (2021)
- [VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild](https://mlsanity.com/paper/2211.14758/) (2022)
- [Face Forensics in the Wild](https://mlsanity.com/paper/2103.16076/) (2021)
- [Combining EfficientNet and Vision Transformers for Video Deepfake Detection](https://mlsanity.com/paper/2107.02612/) (2021)
- [KoDF: A Large-scale Korean DeepFake Detection Dataset](https://mlsanity.com/paper/2103.10094/) (2021)
- [FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion](https://mlsanity.com/paper/2210.15418/) (2022)
- [CORE: Consistent Representation Learning for Face Forgery Detection](https://mlsanity.com/paper/2206.02749/) (2022)
- [Use of a Capsule Network to Detect Fake Images and Videos](https://mlsanity.com/paper/1910.12467/) (2019)
- [Countering Malicious DeepFakes: Survey, Battleground, and Horizon](https://mlsanity.com/paper/2103.00218/) (2021)
- [Adversarial Threats to DeepFake Detection: A Practical Perspective](https://mlsanity.com/paper/2011.09957/) (2020)
- [Exploring Frequency Adversarial Attacks for Face Forgery Detection](https://mlsanity.com/paper/2203.15674/) (2022)
- [Evading DeepFake Detectors via Adversarial Statistical Consistency](https://mlsanity.com/paper/2304.11670/) (2023)
- [TTS Skins: Speaker Conversion via ASR](https://mlsanity.com/paper/1904.08983/) (2019)

## Built on by (most cited first, 16 of 16)

- [Deepfake Detection that Generalizes Across Benchmarks](https://mlsanity.com/paper/2508.06248/) (2025)
- [TalkingHeadBench: A Multi-Modal Benchmark & Analysis of Talking-Head DeepFake Detection](https://mlsanity.com/paper/2505.24866/) (2025)
- [AV-Deepfake1M++: A Large-Scale Audio-Visual Deepfake Benchmark with Real-World Perturbations](https://mlsanity.com/paper/2507.20579/) (2025)
- [Multilingual Source Tracing of Speech Deepfakes: A First Benchmark](https://mlsanity.com/paper/2508.04143/) (2025)
- [Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges](https://mlsanity.com/paper/2406.06965/) (2024)
- [Gen-AI for User Safety: A Survey](https://mlsanity.com/paper/2411.06606/) (2024)
- [Hindi audio-video-Deepfake (HAV-DF): A Hindi language-based Audio-video Deepfake Dataset](https://mlsanity.com/paper/2411.15457/) (2024)
- [FauForensics: Boosting Audio-Visual Deepfake Detection with Facial Action Units](https://mlsanity.com/paper/2505.08294/) (2025)
- [DeepFake Doctor: Diagnosing and Treating Audio-Video Fake Detection](https://mlsanity.com/paper/2506.05851/) (2025)
- [Alethia: A Foundational Encoder for Voice Deepfakes](https://mlsanity.com/paper/2605.00251/) (2026)
- [BioLip: Language-Generalizable Lip-Sync Deepfake Detection via Biomechanical Constraint Violation Modeling](https://mlsanity.com/paper/2604.16808/) (2026)
- [Beyond Seeing Is Believing: On Crowdsourced Detection of Audiovisual Deepfakes](https://mlsanity.com/paper/2605.04797/) (2026)
- [The Alpha Blending Hypothesis: Compositing Shortcut in Deepfake Detection](https://mlsanity.com/paper/2605.10334/) (2026)
- [From Talking to Singing: A New Challenge for Audio-Visual Deepfake Detection](https://mlsanity.com/paper/2605.27944/) (2026)
- [Teacher-Student Structure for Domain Adaptation in Ensemble Audio-Visual Video Deepfake Detection](https://mlsanity.com/paper/2606.15117/) (2026)
- [DF-MoE: Generalizable Deepfake Detection via Multimodal Sparse Mixture-of-Experts](https://mlsanity.com/paper/2608.23363/) (2026)

## Abstract (arXiv)

With the rapid advancement of generative AI, multimodal deepfakes, which manipulate both audio and visual modalities, have drawn increasing public concern. Currently, deepfake detection has emerged as a crucial strategy in countering these growing threats. However, as a key factor in training and validating deepfake detectors, most existing deepfake datasets primarily focus on the visual modal, and the few that are multimodal employ outdated techniques, and their audio content is limited to a single language, thereby failing to represent the cutting-edge advancements and globalization trends in current deepfake technologies. To address this gap, we propose a novel, multilingual, and multimodal deepfake dataset: PolyGlotFake. It includes content in seven languages, created using a variety of cutting-edge and popular Text-to-Speech, voice cloning, and lip-sync technologies. We conduct comprehensive experiments using state-of-the-art detection methods on PolyGlotFake dataset. These experiments demonstrate the dataset's significant challenges and its practical value in advancing research into multimodal deepfake detection.