# Multi-Task Learning in Utterance-Level and Segmental-Level Spoof Detection

arXiv: https://arxiv.org/abs/2107.14132 · HTML: https://mlsanity.com/paper/2107.14132/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2021.

## Built on by (most cited first, 16 of 16)

- [An Efficient Temporary Deepfake Location Approach Based Embeddings for Partially Spoofed Audio Detection](https://mlsanity.com/paper/2309.03036/) (2023)
- [Attack Agnostic Dataset: Towards Generalization and Stabilization of Audio DeepFake Detection](https://mlsanity.com/paper/2206.13979/) (2022)
- [SpecRNet: Towards Faster and More Accessible Audio DeepFake Detection](https://mlsanity.com/paper/2210.06105/) (2022)
- [The defender's perspective on automatic speaker verification: An overview](https://mlsanity.com/paper/2305.12804/) (2023)
- [FairSSD: Understanding Bias in Synthetic Speech Detectors](https://mlsanity.com/paper/2404.10989/) (2024)
- [Enhancing Partially Spoofed Audio Localization with Boundary-aware Attention Mechanism](https://mlsanity.com/paper/2407.21611/) (2024)
- [Range-Based Equal Error Rate for Spoof Localization](https://mlsanity.com/paper/2305.17739/) (2023)
- [KLASSify to Verify: Audio-Visual Deepfake Detection Using SSL-based Audio and Handcrafted Visual Features](https://mlsanity.com/paper/2508.07337/) (2025)
- [TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Models](https://mlsanity.com/paper/2604.01083/) (2026)
- [Catch You and I Can: Revealing Source Voiceprint Against Voice Conversion](https://mlsanity.com/paper/2302.12434/) (2023)
- [Analyzing the Impact of Splicing Artifacts in Partially Fake Speech Signals](https://mlsanity.com/paper/2408.13784/) (2024)
- [Spoof Diarization: "What Spoofed When" in Partially Spoofed Audio](https://mlsanity.com/paper/2406.07816/) (2024)
- [NE-PADD: Leveraging Named Entity Knowledge for Robust Partial Audio Deepfake Detection via Attention Aggregation](https://mlsanity.com/paper/2509.03829/) (2025)
- [An Unsupervised Domain Adaptation Method for Locating Manipulated Region in partially fake Audio](https://mlsanity.com/paper/2407.08239/) (2024)
- [Split and Conquer Partial Deepfake Speech](https://mlsanity.com/paper/2604.02913/) (2026)
- [Robust Spoofed Speech Detection via Temporal Pyramid Modeling](https://mlsanity.com/paper/2606.16837/) (2026)

## Abstract (arXiv)

In this paper, we provide a series of multi-tasking benchmarks for simultaneously detecting spoofing at the segmental and utterance levels in the PartialSpoof database. First, we propose the SELCNN network, which inserts squeeze-and-excitation (SE) blocks into a light convolutional neural network (LCNN) to enhance the capacity of hidden feature selection. Then, we implement multi-task learning (MTL) frameworks with SELCNN followed by bidirectional long short-term memory (Bi-LSTM) as the basic model. We discuss MTL in PartialSpoof in terms of architecture (uni-branch/multi-branch) and training strategies (from-scratch/warm-up) step-by-step. Experiments show that the multi-task model performs relatively better than single-task models. Also, in MTL, a binary-branch architecture more adequately utilizes information from two levels than a uni-branch model. For the binary-branch architecture, fine-tuning a warm-up model works better than training from scratch. Models can handle both segment-level and utterance-level predictions simultaneously overall under a binary-branch multi-task architecture. Furthermore, the multi-task model trained by fine-tuning a segmental warm-up model performs relatively better at both levels except on the evaluation set for segmental detection. Segmental detection should be explored further.