# Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis

arXiv: https://arxiv.org/abs/2305.12709 · HTML: https://mlsanity.com/paper/2305.12709/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 56 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models](https://mlsanity.com/paper/2012.15613/) (2020)
- [Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT](https://mlsanity.com/paper/1904.09077/) (2019)
- [Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems](https://mlsanity.com/paper/1805.04508/) (2018)
- [Characterising Bias in Compressed Models](https://mlsanity.com/paper/2010.03058/) (2020)
- [Intrinsic Bias Metrics Do Not Correlate with Application Bias](https://mlsanity.com/paper/2012.15859/) (2020)
- [Lifting the Curse of Multilinguality by Pre-training Modular Transformers](https://mlsanity.com/paper/2205.06266/) (2022)
- [A Survey Of Cross-lingual Word Embedding Models](https://mlsanity.com/paper/1706.04902/) (2017)
- [The Multilingual Amazon Reviews Corpus](https://mlsanity.com/paper/2010.02573/) (2020)
- [The MultiBERTs: BERT Reproductions for Robustness Analysis](https://mlsanity.com/paper/2106.16163/) (2021)
- [BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance](https://mlsanity.com/paper/1911.02969/) (2019)
- [What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus](https://mlsanity.com/paper/2105.02732/) (2021)
- [A Survey of Race, Racism, and Anti-Racism in NLP](https://mlsanity.com/paper/2106.11410/) (2021)
- [Multilingual Offensive Language Identification with Cross-lingual Embeddings](https://mlsanity.com/paper/2010.05324/) (2020)
- [Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research](https://mlsanity.com/paper/2005.00357/) (2020)
- [Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00699/) (2020)
- [MultiFiT: Efficient Multi-lingual Language Model Fine-tuning](https://mlsanity.com/paper/1909.04761/) (2019)
- [How does Grammatical Gender Affect Noun Representations in Gender-Marking Languages?](https://mlsanity.com/paper/1910.14161/) (2019)
- [How Gender Debiasing Affects Internal Model Representations, and Why It Matters](https://mlsanity.com/paper/2204.06827/) (2022)
- [Grammatical gender associations outweigh topical gender bias in crosslinguistic word embeddings](https://mlsanity.com/paper/2005.08864/) (2020)
- [LOGAN: Local Group Bias Detection by Clustering](https://mlsanity.com/paper/2010.02867/) (2020)
- [Bias Beyond English: Counterfactual Tests for Bias in Sentiment Analysis in Four Languages](https://mlsanity.com/paper/2305.11673/) (2023)

## Built on by (most cited first, 3 of 3)

- [Sociodemographic Bias in Language Models: A Survey and Forward Path](https://mlsanity.com/paper/2306.08158/) (2023)
- [Pruning Multilingual Large Language Models for Multilingual Inference](https://mlsanity.com/paper/2409.16911/) (2024)
- [Analyzing Language Bias Between French and English in Conventional Multilingual Sentiment Analysis Models](https://mlsanity.com/paper/2405.06692/) (2024)

## Abstract (arXiv)

Sentiment analysis (SA) systems are widely deployed in many of the world's languages, and there is well-documented evidence of demographic bias in these systems. In languages beyond English, scarcer training data is often supplemented with transfer learning using pre-trained models, including multilingual models trained on other languages. In some cases, even supervision data comes from other languages. Does cross-lingual transfer also import new biases? To answer this question, we use counterfactual evaluation to test whether gender or racial biases are imported when using cross-lingual transfer, compared to a monolingual transfer setting. Across five languages, we find that systems using cross-lingual transfer usually become more biased than their monolingual counterparts. We also find racial biases to be much more prevalent than gender biases. To spur further research on this topic, we release the sentiment models we used for this study, and the intermediate checkpoints throughout training, yielding 1,525 distinct models; we also release our evaluation code.