mlsanityData to 14 Oct 2026

Preservation of Language Understanding Capabilities in Speech-aware Large Language Models

arXiv 2509.12171 ↗First posted 15 Sep 2025cs.CL cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

The paper presents C3T (Cross-modal Capabilities Conservation Test), a new benchmark for assessing the performance of speech-aware large language models. The benchmark utilizes textual tasks and a voice cloning text-to-speech model to quantify the extent to which language understanding capabilities are preserved when the model is accessed via speech input. C3T quantifies the fairness of the model for different categories of speakers and its robustness across text and speech modalities.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 29 references that are in the map. First posted on arXiv 15 Sep 2025.

Builds on

24 of 29 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  10. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  11. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  12. 2017Counterfactual Fairnesscited by 804
12 more
  1. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  2. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  3. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  4. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
  5. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 243
  6. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 222
  7. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 220
  8. 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 219
  9. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 128
  10. 2024GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speechcited by 30
  11. 2023Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errorscited by 6
  12. 2024USAT: A Universal Speaker-Adaptive Text-to-Speech Approachcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Afrispeech Semantics: Evaluating Audio Semantic Reasoning in Spoken Language Models Across Domains and Accentscited by 2