mlsanityData to 14 Oct 2026

Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation

arXiv 2203.09866 ↗First posted 18 Mar 2022cs.CLcited by 23

Not on the map. Why?

Abstract (arXiv)

Gender bias is largely recognized as a problematic phenomenon affecting language technologies, with recent studies underscoring that it might surface differently across languages. However, most of current evaluation practices adopt a word-level focus on a narrow set of occupational nouns under synthetic conditions. Such protocols overlook key features of grammatical gender languages, which are characterized by morphosyntactic chains of gender agreement, marked on a variety of lexical items and parts-of-speech (POS). To overcome this limitation, we enrich the natural, gender-sensitive MuST-SHE corpus (Bentivogli et al., 2020) with two new linguistic annotation layers (POS and agreement chains), and explore to what extent different lexical categories and agreement phenomena are impacted by gender skews. Focusing on speech translation, we conduct a multifaceted evaluation on three language directions (English-French/Italian/Spanish), with models trained on varying amounts of data and different word segmentation techniques. By shedding light on model behaviours, gender bias, and its detection at several levels of granularity, our findings emphasize the value of dedicated analyses beyond aggregated overall results.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 57 of its 108 references that are in the map. First posted on arXiv 18 Mar 2022.

Builds on

57 of 108 references.

References · 57

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  5. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  7. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  8. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  9. 2018Gender Bias in Coreference Resolutioncited by 442
  10. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  11. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  12. 2019Evaluating Gender Bias in Machine Translationcited by 249
38 more
  1. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 226
  2. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  3. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  4. 2018How2: A Large-scale Dataset for Multimodal Language Understandingcited by 138
  5. 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
  6. 2017Sequence-to-Sequence Models Can Directly Translate Foreign Speechcited by 121
  7. 2021Gender Bias in Machine Translationcited by 117
  8. 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
  9. 2019Getting Gender Right in Neural Machine Translationcited by 98
  10. 2020Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problemcited by 86
  11. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  12. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  13. 2018Pre-training on high-resource speech recognition improves low-resource speech-to-text translationcited by 70
  14. 2019Equalizing Gender Biases in Neural Machine Translation with Word Embeddings Techniquescited by 65
  15. 2021Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translationcited by 62
  16. 2019End-to-End Speech Translation with Knowledge Distillationcited by 58
  17. 2018Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translationcited by 57
  18. 2020Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpuscited by 57
  19. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  20. 2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 54
  21. 2021NeuTral Rewriter: A Rule-Based and Neural Approach to Automatic Rewriting into Gender-Neutral Alternativescited by 46
  22. 2021They, Them, Theirs: Rewriting with Gender-Neutral Englishcited by 44
  23. 2019Examining Gender Bias in Languages with Grammatical Gendercited by 40
  24. 2020Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make Itcited by 40
  25. 2016How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairscited by 39
  26. 2019On the Linguistic Representational Power of Neural Machine Translation Modelscited by 34
  27. 2015It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopediacited by 30
  28. 2019Improving sequence-to-sequence speech recognition training with on-the-fly data augmentationcited by 29
  29. 2021Socially Aware Bias Measurements for Hindi Language Representationscited by 29
  30. 2021Gender Bias Amplification During Speed-Quality Optimization in Neural Machine Translationcited by 28
  31. 2019Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injectioncited by 27
  32. 2020Gender Coreference and Bias Evaluation at WMT 2020cited by 21
  33. 2021Investigating Failures of Automatic Translation in the Case of Unambiguous Gendercited by 20
  34. 2020Evaluating Gender Bias in Speech Translationcited by 19
  35. 2020Breeding Gender-aware Direct Speech Translation Systemscited by 16
  36. 2020Type B Reflexivization as an Unambiguous Testbed for Multilingual Multi-Task Gender Biascited by 15
  37. 2020Decoding and Diversity in Machine Translationcited by 15
  38. 2021Examining Covert Gender Bias: A Case Study in Turkish and English Machine Translation Modelscited by 15

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 8; 2024: 7; 2025: 3; 2026: 1.

23 papers.

Sort:

By area:

  1. 2022MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translationcited by 38
  2. 2023Gender Neutralization for an Inclusive Machine Translation: from Theoretical Foundations to Open Challengescited by 22
  3. 2023Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpuscited by 20
  4. 2023GATE: A Challenge Set for Gender-Ambiguous Translation Examplescited by 18
  5. 2024The "Colonial Impulse" of Natural Language Processing: An Audit of Bengali Sentiment Analysis Tools and Their Identity-based Biasescited by 16
  6. 2022A Study of Gender Impact in Self-supervised Models for Speech-to-Text Systemscited by 15
  7. 2023Recent Advances in Direct Speech-to-text Translationcited by 15
  8. 2024What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Studycited by 15
  9. 2023Target-Agnostic Gender-Aware Contrastive Learning for Mitigating Bias in Multilingual Machine Translationcited by 10
  10. 2022What Do Compressed Multilingual Machine Translation Models Forget?cited by 9
  11. 2024Enhancing Gender-Inclusive Machine Translation with Neomorphemes and Large Language Modelscited by 7
  12. 2023Erasure of Unaligned Attributes from Neural Representationscited by 6
11 more
  1. 2023Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE and INEScited by 6
  2. 2024An Active Learning Framework for Inclusive Generation by Large Language Modelscited by 5
  3. 2025Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translationcited by 5
  4. 2024GATE X-E : A Challenge Set for Gender-Fair Translations from Weakly-Gendered Languagescited by 4
  5. 2024On the Encoding of Gender in Transformer-based ASR Representationscited by 4
  6. 2025Different Speech Translation Models Encode and Translate Speaker Gender Differentlycited by 3
  7. 2025The Unheard Alternative: Contrastive Explanations for Speech-to-Text Modelscited by 3
  8. 2023Good, but not always Fair: An Evaluation of Gender Bias for three commercial Machine Translation Systemscited by 2
  9. 2022The User-Aware Arabic Gender Rewritercited by 1
  10. 2024ARTiST: Automated Text Simplification for Task Guidance in Augmented Realitycited by 0
  11. 2026Bureaucratic Silences: What the Canadian AI Register Reveals, Omits, and Obscurescited by 0