# WMT24 Test Suite: Gender Resolution in Speaker-Listener Dialogue Roles

arXiv: https://arxiv.org/abs/2411.06194 · HTML: https://mlsanity.com/paper/2411.06194/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 10 references that are in the map.
- First posted on arXiv 9 Nov 2024.

## Builds on (most cited first, 8 of 8)

- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Gender Bias in Machine Translation](https://mlsanity.com/paper/2104.06001/) (2021)
- [Gender Coreference and Bias Evaluation at WMT 2020](https://mlsanity.com/paper/2010.06018/) (2020)
- [Hi Guys or Hi Folks? Benchmarking Gender-Neutral Machine Translation with the GeNTE Corpus](https://mlsanity.com/paper/2310.05294/) (2023)
- [What about em? How Commercial Machine Translation Fails to Handle (Neo-)Pronouns](https://mlsanity.com/paper/2305.16051/) (2023)
- [A Prompt Response to the Demand for Automatic Gender-Neutral Translation](https://mlsanity.com/paper/2402.06041/) (2024)
- [Test Suites Task: Evaluation of Gender Fairness in MT with MuST-SHE and INES](https://mlsanity.com/paper/2310.19345/) (2023)
- [Investigating Markers and Drivers of Gender Bias in Machine Translations](https://mlsanity.com/paper/2403.11896/) (2024)

## Built on by (most cited first, 1 of 1)

- [Gender-Neutral Machine Translation Strategies in Practice](https://mlsanity.com/paper/2506.15676/) (2025)

## Abstract (arXiv)

We assess the difficulty of gender resolution in literary-style dialogue settings and the influence of gender stereotypes. Instances of the test suite contain spoken dialogue interleaved with external meta-context about the characters and the manner of speaking. We find that character and manner stereotypes outside of the dialogue significantly impact the gender agreement of referents within the dialogue.