mlsanityData to 7 Oct 2026

Who Are We Talking About? Handling Person Names in Speech Translation

arXiv 2205.06755 ↗First posted 13 May 2022cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent work has shown that systems for speech translation (ST) -- similarly to automatic speech recognition (ASR) -- poorly handle person names. This shortcoming does not only lead to errors that can seriously distort the meaning of the input, but also hinders the adoption of such systems in application scenarios (like computer-assisted interpreting) where the translation of named entities, like person names, is crucial. In this paper, we first analyse the outputs of ASR/ST systems to identify the reasons of failures in person name transcription/translation. Besides the frequency in the training data, we pinpoint the nationality of the referred person as a key factor. We then mitigate the problem by creating multilingual models, and further improve our ST systems by forcing them to jointly generate transcripts and translations, prioritising the former over the latter. Overall, our solutions result in a relative improvement in token-level person name accuracy by 47.8% on average for three language pairs (en->es,fr,it).

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 25 of its 56 references that are in the map. First posted on arXiv 13 May 2022.

Builds on

25 of 56 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  8. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  9. 2017Six Challenges for Neural Machine Translationcited by 334
  10. 2020fairseq S2T: Fast Speech-to-Text Modeling with fairseqcited by 174
  11. 2017Sequence-to-Sequence Models Can Directly Translate Foreign Speechcited by 121
  12. 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
13 more
  1. 2016Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translationcited by 113
  2. 2018Tied Multitask Learning for Neural Speech Translationcited by 54
  3. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  4. 2020Using Synthetic Audio to Improve The Recognition of Out-Of-Vocabulary Words in End-To-End ASR Systemscited by 41
  5. 2022Identifying Weaknesses in Machine Translation Metrics Through Minimum Bayes Risk Decoding: A Case Study for COMETcited by 41
  6. 2020Consistent Transcription and Translation of Speechcited by 14
  7. 2020Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translationcited by 12
  8. 2020On Target Segmentation for Direct Speech Translationcited by 12
  9. 2018End-to-end named entity extraction from speechcited by 11
  10. 2021Is "moby dick" a Whale or a Bird? Named Entities and Terminology in Speech Translationcited by 11
  11. 2021Towards the evaluation of automatic simultaneous speech translation from a communicative perspectivecited by 8
  12. 2018Automatic Estimation of Simultaneous Interpreter Performancecited by 3
  13. 2022KUDO Interpreter Assist: Automated Real-time Support for Remote Interpretationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 1; 2026: 1.

8 papers.

  1. 2023Recent Advances in Direct Speech-to-text Translationcited by 15
  2. 2023Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignmentscited by 6
  3. 2022Joint Speech Translation and Named Entity Recognitioncited by 5
  4. 2022Named Entity Detection and Injection for Direct Speech Translationcited by 4
  5. 2024Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approachcited by 3
  6. 2023Robustness of Multi-Source MT to Transcription Errorscited by 2
  7. 2025PHRASED: Phrase Dictionary Biasing for Speech Translationcited by 1
  8. 2026LOGIC: Efficient and Robust Contextual Biasing for Speech LLMs via Logit-Space Integrationcited by 0