# TIMIT-TTS: a Text-to-Speech Dataset for Multimodal Synthetic Media Detection

arXiv: https://arxiv.org/abs/2209.08000 · HTML: https://mlsanity.com/paper/2209.08000/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2022.

## Built on by (most cited first, 16 of 16)

- [The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audio](https://mlsanity.com/paper/2405.04880/) (2024)
- [Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning](https://mlsanity.com/paper/2412.00175/) (2024)
- [DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Dataset](https://mlsanity.com/paper/2409.08731/) (2024)
- [Leveraging Mixture of Experts for Improved Speech Deepfake Detection](https://mlsanity.com/paper/2409.16077/) (2024)
- [Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization](https://mlsanity.com/paper/2408.01532/) (2024)
- [STOPA: A Database of Systematic VariaTion Of DeePfake Audio for Open-Set Source Tracing and Attribution](https://mlsanity.com/paper/2505.19644/) (2025)
- [RU-AI: A Large Multimodal Dataset for Machine-Generated Content Detection](https://mlsanity.com/paper/2406.04906/) (2024)
- [DiffSSD: A Diffusion-Based Dataset For Speech Forensics](https://mlsanity.com/paper/2409.13049/) (2024)
- [Anomaly Detection and Localization for Speech Deepfakes via Feature Pyramid Matching](https://mlsanity.com/paper/2503.18032/) (2025)
- [DeepfakeBench-MM: A Comprehensive Benchmark for Multimodal Deepfake Detection](https://mlsanity.com/paper/2510.22622/) (2025)
- [The Impact of Audio Watermarking on Audio Anti-Spoofing Countermeasures](https://mlsanity.com/paper/2509.20736/) (2025)
- [AUDDT: A Unified Benchmark Toolkit for Audio and Speech Deepfake Detectors](https://mlsanity.com/paper/2509.21597/) (2025)
- [Alethia: A Foundational Encoder for Voice Deepfakes](https://mlsanity.com/paper/2605.00251/) (2026)
- [Ethical and Technical Limits of Deepfake Speech Datasets](https://mlsanity.com/paper/2606.10911/) (2026)
- [HISPASpoof: A New Dataset For Spanish Speech Forensics](https://mlsanity.com/paper/2509.09155/) (2025)
- [Detecting Audio Deepfakes on the Edge:Lightweight SSL-Based Detection in a Browser Plugin](https://mlsanity.com/paper/2606.30780/) (2026)

## Abstract (arXiv)

With the rapid development of deep learning techniques, the generation and counterfeiting of multimedia material are becoming increasingly straightforward to perform. At the same time, sharing fake content on the web has become so simple that malicious users can create unpleasant situations with minimal effort. Also, forged media are getting more and more complex, with manipulated videos that are taking the scene over still images. The multimedia forensic community has addressed the possible threats that this situation could imply by developing detectors that verify the authenticity of multimedia objects. However, the vast majority of these tools only analyze one modality at a time. This was not a problem as long as still images were considered the most widely edited media, but now, since manipulated videos are becoming customary, performing monomodal analyses could be reductive. Nonetheless, there is a lack in the literature regarding multimodal detectors, mainly due to the scarsity of datasets containing forged multimodal data to train and test the designed algorithms. In this paper we focus on the generation of an audio-visual deepfake dataset. First, we present a general pipeline for synthesizing speech deepfake content from a given real or fake video, facilitating the creation of counterfeit multimodal material. The proposed method uses Text-to-Speech (TTS) and Dynamic Time Warping techniques to achieve realistic speech tracks. Then, we use the pipeline to generate and release TIMIT-TTS, a synthetic speech dataset containing the most cutting-edge methods in the TTS field. This can be used as a standalone audio dataset, or combined with other state-of-the-art sets to perform multimodal research. Finally, we present numerous experiments to benchmark the proposed dataset in both mono and multimodal conditions, showing the need for multimodal forensic detectors and more suitable data.