mlsanityData to 14 Oct 2026

SpeechQE: Estimating the Quality of Direct Speech Translation

arXiv 2410.21485 ↗First posted 28 Oct 2024cs.CLcited by 10

Not on the map. Why?

Abstract (arXiv)

Recent advances in automatic quality estimation for machine translation have exclusively focused on written language, leaving the speech modality underexplored. In this work, we formulate the task of quality estimation for speech translation (SpeechQE), construct a benchmark, and evaluate a family of systems based on cascaded and end-to-end architectures. In this process, we introduce a novel end-to-end system leveraging pre-trained text LLM. Results suggest that end-to-end approaches are better suited to estimating the quality of direct speech translation than using quality estimation systems designed for text in cascaded systems. More broadly, we argue that quality estimation of speech translation needs to be studied as a separate problem from that of text, and release our data and models to guide further research in this space.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 29 of its 47 references that are in the map. First posted on arXiv 28 Oct 2024.

Builds on

29 of 47 references.

References · 29

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  6. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  7. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  8. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  9. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  10. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  11. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  12. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
17 more
  1. 2020fairseq S2T: Fast Speech-to-Text Modeling with fairseqcited by 176
  2. 2023On decoder-only architecture for speech-to-text and large language model integrationcited by 145
  3. 2023Prompting Large Language Models with Speech Recognition Abilitiescited by 140
  4. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 108
  5. 2023Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Taskcited by 88
  6. 2023Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Modelscited by 87
  7. 2023BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writingcited by 79
  8. 2023SLM: Bridge the thin gap between speech and text foundation modelscited by 67
  9. 2019OpenKiwi: An Open Source Framework for Quality Estimationcited by 49
  10. 2023COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learningcited by 31
  11. 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
  12. 2023Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errorscited by 22
  13. 2022BLASER: A Text-Free Speech-to-Speech Translation Evaluation Metriccited by 16
  14. 2023AVFormer: Injecting Vision into Frozen Speech Models for Zero-Shot AV-ASRcited by 16
  15. 2024XLAVS-R: Cross-Lingual Audio-Visual Speech Representation Learning for Noise-Robust Speech Perceptioncited by 11
  16. 2024Evaluating the IWSLT2023 Speech Translation Tasks: Human Annotations, Automatic Metrics, and Segmentationcited by 5
  17. 2023Audio-visual fine-tuning of audio-only ASR modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 5.

10 papers.

  1. 2025Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMscited by 7
  2. 2025An Interdisciplinary Approach to Human-Centered Machine Translationcited by 6
  3. 2025How to Evaluate Speech Translation with Source-Aware Neural MT Metricscited by 3
  4. 2025Early-Exit and Instant Confidence Translation Quality Estimationcited by 2
  5. 2026Why We Need Speech to Evaluate Speech Translationcited by 1
  6. 2025Toward Machine Interpreting: Lessons from Human Interpreting Studiescited by 0
  7. 2026Automatic Labelling of Speech Translation Errorscited by 0
  8. 2026Ouvia: A User-centered Framework for Measuring Usability of Speech Translation in Real-World Communication Scenarioscited by 0
  9. 2026HydraQE: OSU's Submission for the IWSLT 2026 Speech Translation Metrics Shared Taskcited by 0
  10. 2026Measuring User's Mental Models of Speech Translation in Human-AI Collaborationcited by 0