mlsanityData to 14 Oct 2026

How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translation

arXiv 2310.15114 ↗First posted 23 Oct 2023cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

When translating from notional gender languages (e.g., English) into grammatical gender languages (e.g., Italian), the generated translation requires explicit gender assignments for various words, including those referring to the speaker. When the source sentence does not convey the speaker's gender, speech translation (ST) models either rely on the possibly-misleading vocal traits of the speaker or default to the masculine gender, the most frequent in existing training corpora. To avoid such biased and not inclusive behaviors, the gender assignment of speaker-related expressions should be guided by externally-provided metadata about the speaker's gender. While previous work has shown that the most effective solution is represented by separate, dedicated gender-specific models, the goal of this paper is to achieve the same results by integrating the speaker's gender metadata into a single "multi-gender" neural ST model, easier to maintain. Our experiments demonstrate that a single multi-gender model outperforms gender-specialized ones when trained from scratch (with gender accuracy gains up to 12.9 for feminine forms), while fine-tuning from existing ST models does not lead to competitive results.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 35 of its 45 references that are in the map. First posted on arXiv 23 Oct 2023.

Builds on

35 of 45 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Generative Adversarial Networkscited by 5,627
  3. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  4. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  7. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  8. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  9. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  10. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  11. 2020fairseq S2T: Fast Speech-to-Text Modeling with fairseqcited by 176
  12. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 170
23 more
  1. 2017Data Decisions and Theoretical Implications when Adversarially Learning Fair Representationscited by 165
  2. 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
  3. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  4. 2021Gender Bias in Machine Translationcited by 117
  5. 2021The Multilingual TEDx Corpus for Speech Recognition and Translationcited by 101
  6. 2022Linear Adversarial Concept Erasurecited by 101
  7. 2016Toward Multilingual Neural Machine Translation with Universal Encoder and Decodercited by 76
  8. 2019On Measuring Gender Bias in Translation of Gender-neutral Pronounscited by 60
  9. 2020Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpuscited by 57
  10. 2021Diverse Adversaries for Mitigating Bias in Trainingcited by 48
  11. 2022Kernelized Concept Erasurecited by 48
  12. 2016Zero-Resource Translation with Multi-Lingual Neural Machine Translationcited by 46
  13. 2019Multilingual End-to-End Speech Translationcited by 37
  14. 2019One-To-Many Multilingual End-to-end Speech Translationcited by 17
  15. 2020Breeding Gender-aware Direct Speech Translation Systemscited by 16
  16. 2023When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLPcited by 14
  17. 2020On Target Segmentation for Direct Speech Translationcited by 12
  18. 2020On Knowledge Distillation for Direct Speech Translationcited by 11
  19. 2023NAVER LABS Europe's Multilingual Speech Translation Systems for the IWSLT 2023 Low-Resource Trackcited by 9
  20. 2022Mitigating Gender Bias in Machine Translation through Adversarial Learningcited by 8
  21. 2023KIT's Multilingual Speech Translation System for IWSLT 2023cited by 7
  22. 2023Erasure of Unaligned Attributes from Neural Representationscited by 6
  23. 2023No Pitch Left Behind: Addressing Gender Unbalance in Automatic Speech Recognition through Pitch Manipulationcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Conditioning LLMs with Emotion in Neural Machine Translationcited by 1
  2. 2024Usefulness of Emotional Prosody in Neural Machine Translationcited by 0