# Voice Evaluation of Reasoning Ability: Diagnosing the Modality-Induced Performance Gap

arXiv: https://arxiv.org/abs/2509.26542 · HTML: https://mlsanity.com/paper/2509.26542/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2025.

## Built on by (most cited first, 8 of 8)

- [LTS-VoiceAgent: A Listen-Think-Speak Framework for Efficient Streaming Voice Interaction via Semantic Triggering and Incremental Reasoning](https://mlsanity.com/paper/2601.19952/) (2026)
- [AsyncVoice Agent: Real-Time Explanation for LLM Planning and Reasoning](https://mlsanity.com/paper/2510.16156/) (2025)
- [Thinking While Speaking: Inference-Time Knowledge Transfer for Responsive and Intelligent Conversational Voice Agents](https://mlsanity.com/paper/2511.07397/) (2025)
- [MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant](https://mlsanity.com/paper/2609.13076/) (2026)
- [Bridging the Perception Gap: A Lightweight Coarse-to-Fine Architecture for Edge Audio Systems](https://mlsanity.com/paper/2601.15676/) (2026)
- [Entity Binding Failures in Speech LLM Reasoning: Diagnosis and Chain-of-Thought Intervention](https://mlsanity.com/paper/2606.04474/) (2026)
- [RetroThinker: Enabling Retrospective Thinking in Speech LLMs](https://mlsanity.com/paper/2609.11864/) (2026)
- [Efficient Chain-of-Modality Reasoning via Progressive Compression for Spoken Language Models](https://mlsanity.com/paper/2607.19932/) (2026)

## Abstract (arXiv)

We present Voice Evaluation of Reasoning Ability (VERA), a benchmark for evaluating reasoning ability in voice-interactive systems under real-time conversational constraints. VERA comprises 2,931 voice-native episodes derived from established text benchmarks and organized into five tracks (Math, Web, Science, Long-Context, Factual). Each item is adapted for speech interaction while preserving reasoning difficulty. VERA enables direct text-voice comparison within model families and supports analysis of how architectural choices affect reliability. We assess 12 contemporary voice systems alongside strong text baselines and observe large, consistent modality gaps: on competition mathematics a leading text model attains 74.8% accuracy while its voice counterpart reaches 6.1%; macro-averaged across tracks the best text models achieve 54.0% versus 11.3% for voice. Latency-accuracy analyses reveal a low-latency plateau, where fast voice systems cluster around ~10% accuracy, while approaching text performance requires sacrificing real-time interaction. Diagnostic experiments indicate that common mitigations are insufficient. Increasing "thinking time" yields negligible gains; a decoupled cascade that separates reasoning from narration improves accuracy but still falls well short of text and introduces characteristic grounding/consistency errors. Failure analyses further show distinct error signatures across native streaming, end-to-end, and cascade designs. VERA provides a reproducible testbed and targeted diagnostics for architectures that decouple thinking from speaking, offering a principled way to measure progress toward real-time voice assistants that are both fluent and reliably reasoned.