# Unified Shared Encoder in Spoof-Aware Speaker Verification with Hybrid Wavelet Prompt Tuning

arXiv: https://arxiv.org/abs/2610.08845 · HTML: https://mlsanity.com/paper/2610.08845/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Anti-spoofing and adversarial audio attacks, one of 11 directions in Speech and audio language models.
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Placed in

- Speech and audio language models / [Anti-spoofing and adversarial audio attacks](https://mlsanity.com/speech/anti-spoofing-and-adversarial-audio-attacks/)

## Builds on (most cited first, 20 of 20)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale](https://mlsanity.com/paper/2408.08739/) (2024)
- [A Survey on Speech Deepfake Detection](https://mlsanity.com/paper/2404.13914/) (2024)
- [SpoofCeleb: Speech Deepfake Detection and SASV In The Wild](https://mlsanity.com/paper/2409.17285/) (2024)
- [An attention-based backend allowing efficient fine-tuning of transformer models for speaker verification](https://mlsanity.com/paper/2210.01273/) (2022)
- [Frequency and Multi-Scale Selective Kernel Attention for Speaker Verification](https://mlsanity.com/paper/2204.01005/) (2022)
- [Detect All-Type Deepfake Audio: Wavelet Prompt Tuning for Enhanced Auditory Perception](https://mlsanity.com/paper/2504.06753/) (2025)
- [Generalized Spoofing Detection Inspired from Audio Generation Artifacts](https://mlsanity.com/paper/2104.04111/) (2021)
- [Direct Speech-to-Speech Neural Machine Translation: A Survey](https://mlsanity.com/paper/2411.14453/) (2024)
- [WildSpoof Challenge Evaluation Plan](https://mlsanity.com/paper/2508.16858/) (2025)
- [WaveSP-Net: Learnable Wavelet-Domain Sparse Prompt Tuning for Speech Deepfake Detection](https://mlsanity.com/paper/2510.05305/) (2025)
- [Joint Optimization of Speaker and Spoof Detectors for Spoofing-Robust Automatic Speaker Verification](https://mlsanity.com/paper/2510.01818/) (2025)
- [BUT Systems for WildSpoof Challenge: SASV in the Wild](https://mlsanity.com/paper/2512.12851/) (2025)
- [Spoofing-Aware Speaker Verification via Wavelet Prompt Tuning and Multi-Model Ensembles](https://mlsanity.com/paper/2601.17557/) (2026)

## Abstract (arXiv)

Spoofing-aware speaker verification (SASV) must confirm who is speaking and that the speech is genuine, but current systems gain one at the expense of the other. Modular cascades are accurate yet run two large self-supervised encoders, whereas end-to-end models are efficient but lose accuracy when a single embedding has to serve two conflicting objectives. We show that this trade-off between efficiency and specialization is avoidable. A single frozen W2V-BERT~2.0 backbone is adapted per task by deep hybrid Wavelet Prompt Tuning (WPT), so each branch obtains its own view of the shared encoder through dedicated prompts and a task head, and the scores are combined only at inference. Training only about 7M parameters, a small fraction of those a strong two-encoder cascade requires, the system surpasses it on SpoofCeleb evaluation with 0.03% CM-EER and 0.038 min a-DCF against 0.16% and 0.047. Because the backbone is shared and frozen, new branches attach without retraining the heads and prompts already in place. On ASVspoof5, with adversarial attacks and codec distortions, it reaches 0.092 min a-DCF and 3.60% CM-EER using only the provided data, indicating that the design holds under realistic in-the-wild threats.