Who Receives Respect? A Counterfactual Benchmark for Honorific Allocation Bias in Bengali
Not on the map. Why?
Abstract (arXiv)
Bengali has no grammatical gender, but honorificity is obligatory: every third-person reference must choose ordinary or respectful register. English has no such signal, so the system must choose a level of deference that the source never specified. We use this property to measure how large language models (LLMs) allocate respect in English--Bengali translation. Our evaluation set contains 2,016 sentences across 72 occupations. Each appears in seven sentence frames under four pronoun conditions. Masculine and feminine versions differ by one word, forming counterfactual pairs. We report a single Honorific Bias Score (HBS) for each model on a 0--100 scale, covering both the gender and occupation axes. A lower score means that respect is allocated more evenly. For the occupation analysis, we compute each occupation's honorific rate across the seven no-pronoun sentence frames. The standard deviation of these 72 occupation-level rates ranges from 17.2 to 35.5 percentage points across models, showing substantial variation in how honorifics are allocated across occupations. Occupation is associated with 3.2% to 51.5% of the variance in register choice, compared with at most 1.2% for gender. Gender effects are much smaller overall: only two of the nine models show significant directional gender gaps, and they favor opposite genders. HBS ranges from 19.9 to 70.1 across the nine models, showing substantial variation in how evenly the models allocate respect.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 14 references that are in the map. First posted on arXiv 10 Oct 2026.
Builds on
9 of 14 references.
References · 9
- 2018Gender Bias in Coreference Resolutioncited by 442
- 2019Evaluating Gender Bias in Machine Translationcited by 249
- 2021Gender Bias in Machine Translationcited by 117
- 2019On Measuring Gender Bias in Translation of Gender-neutral Pronounscited by 60
- 2023ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languagescited by 34
- 2025Religious Bias Landscape in Language and Text-to-Image Models: Analysis, Detection, and Debiasing Strategiescited by 9
- 2025BharatBBQ: A Multilingual Bias Benchmark for Question Answering in the Indian Contextcited by 6
- 2024An Empirical Study of Gendered Stereotypes in Emotional Attributes for Bangla in Multilingual Large Language Modelscited by 5
- 2023Politeness Stereotypes and Attack Vectors: Gender Stereotypes in Japanese and Korean Language Modelscited by 1
Built on by
Not cited yet.