# Deep Spectro-temporal Artifacts for Detecting Synthesized Speech

arXiv: https://arxiv.org/abs/2210.05254 · HTML: https://mlsanity.com/paper/2210.05254/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 11 Oct 2022.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021)
- [Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation](https://mlsanity.com/paper/2202.12233/) (2022)
- [AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines](https://mlsanity.com/paper/2010.11567/) (2020)
- [ADD 2022: the First Audio Deep Synthesis Detection Challenge](https://mlsanity.com/paper/2202.08433/) (2022)
- [ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech](https://mlsanity.com/paper/2102.05889/) (2021)
- [Investigating self-supervised front ends for speech spoofing countermeasures](https://mlsanity.com/paper/2111.07725/) (2021)
- [Half-Truth: A Partially Fake Audio Detection Dataset](https://mlsanity.com/paper/2104.03617/) (2021)
- [Replay and Synthetic Speech Detection with Res2net Architecture](https://mlsanity.com/paper/2010.15006/) (2020)
- [An Initial Investigation for Detecting Partially Spoofed Audio](https://mlsanity.com/paper/2104.02518/) (2021)
- [Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network](https://mlsanity.com/paper/2104.10122/) (2021)
- [An Empirical Study on Channel Effects for Synthetic Voice Spoofing Countermeasure Systems](https://mlsanity.com/paper/2104.01320/) (2021)
- [UR Channel-Robust Synthetic Speech Detection System for ASVspoof 2021](https://mlsanity.com/paper/2107.12018/) (2021)
- [FastAudio: A Learnable Audio Front-End for Spoof Speech Detection](https://mlsanity.com/paper/2109.02774/) (2021)
- [The DKU Replay Detection System for the ASVspoof 2019 Challenge: On Data Augmentation, Feature Representation, Classification, and Fusion](https://mlsanity.com/paper/1907.02663/) (2019)

## Built on by (most cited first, 2 of 2)

- [Enhancing Partially Spoofed Audio Localization with Boundary-aware Attention Mechanism](https://mlsanity.com/paper/2407.21611/) (2024)
- [An Unsupervised Domain Adaptation Method for Locating Manipulated Region in partially fake Audio](https://mlsanity.com/paper/2407.08239/) (2024)

## Abstract (arXiv)

The Audio Deep Synthesis Detection (ADD) Challenge has been held to detect generated human-like speech. With our submitted system, this paper provides an overall assessment of track 1 (Low-quality Fake Audio Detection) and track 2 (Partially Fake Audio Detection). In this paper, spectro-temporal artifacts were detected using raw temporal signals, spectral features, as well as deep embedding features. To address track 1, low-quality data augmentation, domain adaptation via finetuning, and various complementary feature information fusion were aggregated in our system. Furthermore, we analyzed the clustering characteristics of subsystems with different features by visualization method and explained the effectiveness of our proposed greedy fusion strategy. As for track 2, frame transition and smoothing were detected using self-supervised learning structure to capture the manipulation of PF attacks in the time domain. We ranked 4th and 5th in track 1 and track 2, respectively.