# GENder-IT: An Annotated English-Italian Parallel Challenge Set for Cross-Linguistic Natural Gender Phenomena

arXiv: https://arxiv.org/abs/2108.02854 · HTML: https://mlsanity.com/paper/2108.02854/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 5 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Gender Bias in Coreference Resolution](https://mlsanity.com/paper/1804.09301/) (2018)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Getting Gender Right in Neural Machine Translation](https://mlsanity.com/paper/1909.05088/) (2019)
- [Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem](https://mlsanity.com/paper/2004.04498/) (2020)
- [Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation](https://mlsanity.com/paper/2102.00287/) (2021)
- [Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus](https://mlsanity.com/paper/2006.05754/) (2020)
- [Personalized Machine Translation: Preserving Original Author Traits](https://mlsanity.com/paper/1610.05461/) (2016)
- [Neural Machine Translation Doesn't Translate Gender Coreference Right Unless You Make It](https://mlsanity.com/paper/2010.05332/) (2020)
- [GeBioToolkit: Automatic Extraction of Gender-Balanced Multilingual Corpus of Wikipedia Biographies](https://mlsanity.com/paper/1912.04778/) (2019)

## Built on by (most cited first, 16 of 16)

- [Perturbation Augmentation for Fairer NLP](https://mlsanity.com/paper/2205.12586/) (2022)
- [MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translation](https://mlsanity.com/paper/2211.01355/) (2022)
- [Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpus](https://mlsanity.com/paper/2310.05294/) (2023)
- [What about em? How Commercial Machine Translation Fails to Handle (Neo-)Pronouns](https://mlsanity.com/paper/2305.16051/) (2023)
- [A Tale of Pronouns: Interpretability Informs Gender Bias Mitigation for Fairer Instruction-Tuned Machine Translation](https://mlsanity.com/paper/2310.12127/) (2023)
- [GATE: A Challenge Set for Gender-Ambiguous Translation Examples](https://mlsanity.com/paper/2303.03975/) (2023)
- [What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study](https://mlsanity.com/paper/2410.00545/) (2024)
- [The Arabic Parallel Gender Corpus 2.0: Extensions and Analyses](https://mlsanity.com/paper/2110.09216/) (2021)
- [Angler: Helping Machine Translation Practitioners Prioritize Model Improvements](https://mlsanity.com/paper/2304.05967/) (2023)
- [Watching the Watchers: Exposing Gender Disparities in Machine Translation Quality Estimation](https://mlsanity.com/paper/2410.10995/) (2024)
- [Assumed Identities: Quantifying Gender Bias in Machine Translation of Gender-Ambiguous Occupational Terms](https://mlsanity.com/paper/2503.04372/) (2025)
- [Gender Bias in Machine Translation and The Era of Large Language Models](https://mlsanity.com/paper/2401.10016/) (2024)
- [Data Bias According to Bipol: Men are Naturally Right and It is the Role of Women to Follow Their Lead](https://mlsanity.com/paper/2404.04838/) (2024)
- [Uncertainty Quantification for Evaluating Machine Translation Bias](https://mlsanity.com/paper/2507.18338/) (2025)
- [What Triggers my Model? Contrastive Explanations Inform Gender Choices by Translation Models](https://mlsanity.com/paper/2512.08440/) (2025)
- [ConGA: Guidelines for Contextual Gender Annotation. A Framework for Annotating Gender in Machine Translation](https://mlsanity.com/paper/2603.17962/) (2026)

## Abstract (arXiv)

Languages differ in terms of the absence or presence of gender features, the number of gender classes and whether and where gender features are explicitly marked. These cross-linguistic differences can lead to ambiguities that are difficult to resolve, especially for sentence-level MT systems. The identification of ambiguity and its subsequent resolution is a challenging task for which currently there aren't any specific resources or challenge sets available. In this paper, we introduce gENder-IT, an English--Italian challenge set focusing on the resolution of natural gender phenomena by providing word-level gender tags on the English source side and multiple gender alternative translations, where needed, on the Italian target side.