# Can Audio Large Language Models Verify Speaker Identity?

arXiv: https://arxiv.org/abs/2509.19755 · HTML: https://mlsanity.com/paper/2509.19755/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2025.

## Built on by (most cited first, 2 of 2)

- [VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Models](https://mlsanity.com/paper/2601.19956/) (2026)
- [Large Audio Language Models for Spoofing-Aware Speaker Verification](https://mlsanity.com/paper/2607.14753/) (2026)

## Abstract (arXiv)

This paper investigates adapting Audio Large Language Models (ALLMs) for speaker verification (SV). We reformulate SV as an audio question-answering task and conduct comprehensive zero-shot evaluations on public benchmarks, showing that current ALLMs have limited zero-shot SV capability and often struggle in diverse acoustic conditions. To address this challenge, we perform supervised fine-tuning on speaker verification data. A rule-based hard pair sampling strategy is proposed to construct more challenging training pairs. Lightweight fine-tuning substantially improves the performance, though there is still a gap between ALLMs and conventional models. Then, we extend to text-dependent SV by jointly querying ALLMs to verify speaker identity and spoken content, yielding results competitive with cascaded ASR-SV systems. Our findings demonstrate that with proper adaptation, ALLMs hold substantial potential as a unified model for robust speaker verification systems, while maintaining the general audio understanding capabilities.