mlsanityData to 14 Oct 2026

Text Scores Do Not Establish Performance on Lexically Non-Diagnostic Speech Tasks: A Qwen2-Audio Quantization Case Study

arXiv 2609.26823 ↗First posted 20 Sep 2026cs.SD cs.CL eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Text-output scores alone do not show whether quantization preserves performance on speech tasks whose target labels cannot be recovered from the transcript. We evaluate fixed mixed 4/8-bit Qwen2-Audio-7B-Instruct allocations averaging 6 and 7 bits per parameter on 508 English-to-German FLEURS utterances and on 512 RAVDESS emotion clips from 16 speakers. The BLEU and chrF differences from half precision (FP16) have intervals that include zero for both allocations. On RAVDESS, the same two sentences occur equally often with every emotion label. The absolute accuracy differences from FP16 are -3.71% for 6 bit and -1.17% for 7 bit. The 6-bit speaker interval excludes zero and an exact two-sided sign-flip test gives p=0.0148; the 7-bit interval includes zero. Same-budget controls do not identify either selected allocation as best. This case study shows why translation scores and performance on tasks beyond the transcript need separate evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 18 references that are in the map. First posted on arXiv 20 Sep 2026.

Builds on

11 of 18 references.

References · 11

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  5. 2025Do Audio LLMs Really LISTEN, or Just Transcribe? Measuring Lexical vs. Acoustic Emotion Cues Reliancecited by 24
  6. 2025Efficient Speech Translation through Model Compression and Knowledge Distillationcited by 5
  7. 2026A Closer Look at Failure Modes in Temporal Understanding of Large Audio-Language Modelscited by 2
  8. 2026From Semantics to Readout: Mechanistic Understanding of Audio Tokens after Fine-Tuning for Temporal Audio Groundingcited by 2
  9. 2026Direct or Mediated? Task-Dependent Audio Information Routing in Large Audio Language Modelscited by 1
  10. 2026TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Modelscited by 1
  11. 2026TAG-Bench: Benchmarking Temporal Audio Grounding in Large Audio Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Segmentation-Variant Codebooks for Preservation of Paralinguistic and Prosodic Information
  2. 2026K-Quantization and its Impact on Output Performance
  3. 2026The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs
  4. 2024When Quantization Affects Confidence of Large Language Models?
  5. 2026The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speech
  6. 2026Quantization Degradation in Large Language Models: A Signal-Noise Perspective
  7. 2026Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architectures
  8. 2023AffectEcho: Speaker Independent and Language-Agnostic Emotion and Affect Transfer for Speech Synthesis