# Synthesized Speech Detection Using Convolutional Transformer-Based Spectrogram Analysis

arXiv: https://arxiv.org/abs/2205.01800 · HTML: https://mlsanity.com/paper/2205.01800/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 40 references that are in the map.
- First posted on arXiv 3 May 2022.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Escaping the Big Data Paradigm with Compact Transformers](https://mlsanity.com/paper/2104.05704/) (2021)
- [Deepfakes Detection with Automatic Face Weighting](https://mlsanity.com/paper/2004.12027/) (2020)
- [Neural Style Transfer for Audio Spectograms](https://mlsanity.com/paper/1801.01589/) (2018)
- [We Need No Pixels: Video Manipulation Detection Using Stream Descriptors](https://mlsanity.com/paper/1906.08743/) (2019)

## Built on by (most cited first, 5 of 5)

- [Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook](https://mlsanity.com/paper/2411.19537/) (2024)
- [Transformer-Based Speech Synthesizer Attribution in an Open Set Scenario](https://mlsanity.com/paper/2210.07546/) (2022)
- [Compression Robust Synthetic Speech Detection Using Patched Spectrogram Transformer](https://mlsanity.com/paper/2402.14205/) (2024)
- [Quantum Vision Theory Applied to Audio Classification for Deepfake Speech Detection](https://mlsanity.com/paper/2604.08104/) (2026)
- [DF-MoE: Generalizable Deepfake Detection via Multimodal Sparse Mixture-of-Experts](https://mlsanity.com/paper/2608.23363/) (2026)

## Abstract (arXiv)

Synthesized speech is common today due to the prevalence of virtual assistants, easy-to-use tools for generating and modifying speech signals, and remote work practices. Synthesized speech can also be used for nefarious purposes, including creating a purported speech signal and attributing it to someone who did not speak the content of the signal. We need methods to detect if a speech signal is synthesized. In this paper, we analyze speech signals in the form of spectrograms with a Compact Convolutional Transformer (CCT) for synthesized speech detection. A CCT utilizes a convolutional layer that introduces inductive biases and shared weights into a network, allowing a transformer architecture to perform well with fewer data samples used for training. The CCT uses an attention mechanism to incorporate information from all parts of a signal under analysis. Trained on both genuine human voice signals and synthesized human voice signals, we demonstrate that our CCT approach successfully differentiates between genuine and synthesized speech signals.