mlsanityData to 7 Oct 2026

Assessing the Role of Lexical Semantics in Cross-lingual Transfer through Controlled Manipulations

arXiv 2408.07599 ↗First posted 14 Aug 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

While cross-linguistic model transfer is effective in many settings, there is still limited understanding of the conditions under which it works. In this paper, we focus on assessing the role of lexical semantics in cross-lingual transfer, as we compare its impact to that of other language properties. Examining each language property individually, we systematically analyze how differences between English and a target language influence the capacity to align the language with an English pretrained representation space. We do so by artificially manipulating the English sentences in ways that mimic specific characteristics of the target language, and reporting the effect of each manipulation on the quality of alignment with the representation space. We show that while properties such as the script or word order only have a limited impact on alignment quality, the degree of lexical matching between the two languages, which we define using a measure of translation entropy, greatly affects it.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 41 references that are in the map. First posted on arXiv 14 Aug 2024.

Builds on

24 of 41 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  6. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 738
  7. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  8. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  9. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  10. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  11. 2020SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddingscited by 155
  12. 2019Choosing Transfer Languages for Cross-Lingual Learningcited by 143
12 more
  1. 2021Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processingcited by 68
  2. 2023Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Reviewcited by 50
  3. 2021When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfercited by 41
  4. 2022Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languagescited by 25
  5. 2022Multi Task Learning For Zero Shot Performance Prediction of Multilingual Modelscited by 20
  6. 2022Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structurecited by 17
  7. 2021Analysing The Impact Of Linguistic Features On Cross-Lingual Transfercited by 16
  8. 2023A Crosslingual Investigation of Conceptualization in 1335 Languagescited by 11
  9. 2023Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphscited by 11
  10. 2022Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studiescited by 9
  11. 2021On the Relation between Syntactic Divergence and Zero-Shot Performancecited by 4
  12. 2023Improving Cross-Lingual Transfer through Subtree-Aware Word Reorderingcited by 4

Built on by

Not cited yet.