# Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection

arXiv: https://arxiv.org/abs/2310.15752 · HTML: https://mlsanity.com/paper/2310.15752/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2023.

## Built on by (most cited first, 3 of 3)

- [What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study](https://mlsanity.com/paper/2410.00545/) (2024)
- [Sentence-level Aggregation of Lexical Metrics Correlates Stronger with Human Judgements than Corpus-level Aggregation](https://mlsanity.com/paper/2407.12832/) (2024)
- [The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models](https://mlsanity.com/paper/2509.26543/) (2025)

## Abstract (arXiv)

When translating words referring to the speaker, speech translation (ST) systems should not resort to default masculine generics nor rely on potentially misleading vocal traits. Rather, they should assign gender according to the speakers' preference. The existing solutions to do so, though effective, are hardly feasible in practice as they involve dedicated model re-training on gender-labeled ST data. To overcome these limitations, we propose the first inference-time solution to control speaker-related gender inflections in ST. Our approach partially replaces the (biased) internal language model (LM) implicitly learned by the ST decoder with gender-specific external LMs. Experiments on en->es/fr/it show that our solution outperforms the base models and the best training-time mitigation strategy by up to 31.0 and 1.6 points in gender accuracy, respectively, for feminine forms. The gains are even larger (up to 32.0 and 3.4) in the challenging condition where speakers' vocal traits conflict with their gender.