# CoLMbo: Speaker Language Model for Descriptive Profiling

arXiv: https://arxiv.org/abs/2506.09375 · HTML: https://mlsanity.com/paper/2506.09375/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 11 Jun 2025.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [ClipCap: CLIP Prefix for Image Captioning](https://mlsanity.com/paper/2111.09734/) (2021)
- [SALMONN: Towards Generic Hearing Abilities for Large Language Models](https://mlsanity.com/paper/2310.13289/) (2023)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Pengi: An Audio Language Model for Audio Tasks](https://mlsanity.com/paper/2305.11834/) (2023)
- [Listen, Think, and Understand](https://mlsanity.com/paper/2305.10790/) (2023)
- [GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities](https://mlsanity.com/paper/2406.11768/) (2024)
- [EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2406.06185/) (2024)
- [MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verification](https://mlsanity.com/paper/2203.15249/) (2022)
- [Audio Entailment: Assessing Deductive Reasoning for Audio Understanding](https://mlsanity.com/paper/2407.18062/) (2024)
- [Explainable Attribute-Based Speaker Verification](https://mlsanity.com/paper/2405.19796/) (2024)
- [Speaker Embedding Extraction with Phonetic Information](https://mlsanity.com/paper/1804.04862/) (2018)
- [PDAF: A Phonetic Debiasing Attention Framework For Speaker Verification](https://mlsanity.com/paper/2409.05799/) (2024)
- [SEGAA: A Unified Approach to Predicting Age, Gender, and Emotion in Speech](https://mlsanity.com/paper/2403.00887/) (2024)

## Built on by (most cited first, 5 of 5)

- [SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoning](https://mlsanity.com/paper/2605.15044/) (2026)
- [DELULU: Discriminative Embedding Learning Using Latent Units for Speaker-Aware Self-Trained Speech Foundational Model](https://mlsanity.com/paper/2510.17662/) (2025)
- [Large Audio Language Models for Spoofing-Aware Speaker Verification](https://mlsanity.com/paper/2607.14753/) (2026)
- [AcoustiClaim: A Numeric Claim Benchmark with Instrument Ground Truth](https://mlsanity.com/paper/2609.30483/) (2026)
- [CoLMbo-SV: A Grounded Language Model for Explainable Speaker Verification](https://mlsanity.com/paper/2609.33212/) (2026)

## Abstract (arXiv)

Speaker recognition systems are often limited to classification tasks and struggle to generate detailed speaker characteristics or provide context-rich descriptions. These models primarily extract embeddings for speaker identification but fail to capture demographic attributes such as dialect, gender, and age in a structured manner. This paper introduces CoLMbo, a Speaker Language Model (SLM) that addresses these limitations by integrating a speaker encoder with prompt-based conditioning. This allows for the creation of detailed captions based on speaker embeddings. CoLMbo utilizes user-defined prompts to adapt dynamically to new speaker characteristics and provides customized descriptions, including regional dialect variations and age-related traits. This innovative approach not only enhances traditional speaker profiling but also excels in zero-shot scenarios across diverse datasets, marking a significant advancement in the field of speaker recognition.