mlsanityData to 14 Oct 2026

Is a Prestigious Job the same as a Prestigious Country? A Case Study on Multilingual Sentence Embeddings and European Countries

arXiv 2305.14482 ↗First posted 23 May 2023cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

We study how multilingual sentence representations capture European countries and occupations and how this differs across European languages. We prompt the models with templated sentences that we machine-translate into 12 European languages and analyze the most prominent dimensions in the embeddings.Our analysis reveals that the most prominent feature in the embedding is the geopolitical distinction between Eastern and Western Europe and the country's economic strength in terms of GDP. When prompted specifically for job prestige, the embedding space clearly distinguishes high and low-prestige jobs. The occupational dimension is uncorrelated with the most dominant country dimensions in three out of four studied models. The exception is a small distilled model that exhibits a connection between occupational prestige and country of origin, which is a potential source of nationality-based discrimination. Our findings are consistent across languages.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 29 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

17 of 29 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 981
  6. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  7. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  8. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  9. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  10. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 593
  11. 2019Multilingual Universal Sentence Encoder for Semantic Retrievalcited by 193
  12. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
5 more
  1. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 149
  2. 2023Nationality Bias in Text Generationcited by 50
  3. 2022Bitext Mining Using Distilled Sentence Representations for Low-Resource Languagescited by 48
  4. 2022Speaking Multiple Languages Affects the Moral Bias of Language Modelscited by 34
  5. 2022GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Modelscited by 30

Built on by

Not cited yet.