# Benchmarking Contextual and Paralinguistic Reasoning in Speech-LLMs: A Case Study with In-the-Wild Data

arXiv: https://arxiv.org/abs/2509.16589 · HTML: https://mlsanity.com/paper/2509.16589/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning.
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 20 Sep 2025.

## Placed in

- LLM reasoning / [Audio-language model reasoning](https://mlsanity.com/lr/audio-language-model-reasoning/)

## Builds on (most cited first, 14 of 14)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://mlsanity.com/paper/2311.07919/) (2023)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [SALMONN: Towards Generic Hearing Abilities for Large Language Models](https://mlsanity.com/paper/2310.13289/) (2023)
- [AudioPaLM: A Large Language Model That Can Speak and Listen](https://mlsanity.com/paper/2306.12925/) (2023)
- [BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing](https://mlsanity.com/paper/2309.00916/) (2023)
- [LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPT](https://mlsanity.com/paper/2310.04673/) (2023)
- [A Survey on Speech Large Language Models for Understanding](https://mlsanity.com/paper/2410.18908/) (2024)
- [Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations](https://mlsanity.com/paper/2402.12786/) (2024)
- [Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speech](https://mlsanity.com/paper/2410.01162/) (2024)
- [MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models](https://mlsanity.com/paper/2412.09818/) (2024)

## Built on by (most cited first, 10 of 10)

- [Do Audio LLMs Listen or Read? Analyzing and Mitigating Paralinguistic Failures with VoxParadox](https://mlsanity.com/paper/2605.27772/) (2026)
- [MENASpeechBank: A Reference Voice Bank with Persona-Conditioned Multi-Turn Conversations for AudioLLMs](https://mlsanity.com/paper/2602.07036/) (2026)
- [Language-Aware Distillation for Multilingual Instruction-Following Speech LLMs with ASR-Only Supervision](https://mlsanity.com/paper/2603.07025/) (2026)
- [VoiceLongMemEval: Do Assistants Remember How You Sounded?](https://mlsanity.com/paper/2609.00570/) (2026)
- [MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages](https://mlsanity.com/paper/2511.04914/) (2025)
- [Speech-based Psychological Crisis Assessment using LLMs](https://mlsanity.com/paper/2605.10027/) (2026)
- [Encoder-Side Neuron Identification and Amplification for Acoustic Perception in Large Audio-Language Models](https://mlsanity.com/paper/2607.11801/) (2026)
- [MERaLiON-GR: Speech Gender Recognition Model for English and SEA Languages](https://mlsanity.com/paper/2608.04433/) (2026)
- [Almost Human, Except When It Matters: VoxParity and the Decisions a Voice Should Change](https://mlsanity.com/paper/2609.35922/) (2026)
- [When Does a Spoken Agent Have Enough Evidence to Act? The PACT-SLM Contract Test](https://mlsanity.com/paper/2609.38232/) (2026)

## Abstract (arXiv)

Recent speech-LLMs have shown impressive performance in tasks like transcription and translation, yet they remain limited in understanding the paralinguistic aspects of speech crucial for social and emotional intelligence. We propose CP-Bench, a benchmark for evaluating speech-LLMs on contextual paralinguistic reasoning the integration of verbal content with non-verbal cues like emotion and prosody. The benchmark includes two curated question answering (QA) datasets requiring both linguistic and empathetic understanding. We evaluate state-of-the-art speech-LLMs from both open and closed-source models and perform a comprehensive analysis across different question types. The top two models were further analyzed under temperature tuning to understand its effect on this task. Our benchmark reveals a key gap in existing evaluations and offers insights into building more context-aware and emotionally intelligent speech-capable LLMs.