# Auden-Voice: General-Purpose Voice Encoder for Speech and Language Understanding

arXiv: https://arxiv.org/abs/2511.15145 · HTML: https://mlsanity.com/paper/2511.15145/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2025.

## Built on by (most cited first, 3 of 3)

- [TagSpeech: End-to-End Multi-Speaker ASR and Diarization with Fine-Grained Temporal Grounding](https://mlsanity.com/paper/2601.06896/) (2026)
- [Revisiting Audio-language Pretraining for Learning General-purpose Audio Representation](https://mlsanity.com/paper/2511.16757/) (2025)
- [AzeroS: Extending LLM to Speech with Self-Generated Instruction-Free Tuning](https://mlsanity.com/paper/2601.06086/) (2025)

## Abstract (arXiv)

Human voice encodes both identity and paralinguistic cues, yet encoders in large audio-language models (LALMs) rarely balance both aspects. In this work, we present a study toward building a general-purpose voice encoder that captures nuanced voice cues. Through a comprehensive evaluation, we find that multi-task training yields the most balanced representations, whereas contrastive language-audio pretraining (CLAP) primarily improves retrieval without enhancing paralinguistic understanding. Our final encoder, Auden-Voice, also demonstrates strong performance when integrated with LLMs. The code and training recipes will be released with the audio understanding toolkit Auden.