mlsanityData to 14 Oct 2026

Combating the Curse of Multilinguality in Cross-Lingual WSD by Aligning Sparse Contextualized Word Representations

arXiv 2307.13776 ↗First posted 25 Jul 2023cs.CL cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

In this paper, we advocate for using large pre-trained monolingual language models in cross lingual zero-shot word sense disambiguation (WSD) coupled with a contextualized mapping mechanism. We also report rigorous experiments that illustrate the effectiveness of employing sparse contextualized word representations obtained via a dictionary learning procedure. Our experimental results demonstrate that the above modifications yield a significant improvement of nearly 6.5 points of increase in the average F-score (from 62.0 to 68.5) over a collection of 17 typologically diverse set of target languages. We release our source code for replicating our experiments at https://github.com/begab/sparsity_makes_sense.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 26 of its 54 references that are in the map. First posted on arXiv 25 Jul 2023.

Builds on

26 of 54 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  3. 2018Deep contextualized word representationscited by 2,290
  4. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  5. 2019How multilingual is Multilingual BERT?cited by 645
  6. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  7. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  8. 2019CamemBERT: a Tasty French Language Modelcited by 368
  9. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 318
  10. 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
  11. 2019Visualizing and Measuring the Geometry of BERTcited by 216
  12. 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
14 more
  1. 2013Exploiting Similarities among Languages for Machine Translationcited by 191
  2. 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
  3. 2019Emerging Cross-lingual Structure in Pretrained Language Modelscited by 181
  4. 2019BERTje: A Dutch BERT Modelcited by 109
  5. 2017Offline bilingual word vectors, orthogonal transformations and the inverted softmaxcited by 96
  6. 2018Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterioncited by 70
  7. 2019Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsingcited by 70
  8. 2019Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsingcited by 46
  9. 2019Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguationcited by 40
  10. 2020Give your Text Representation Models some Love: the Case for Basquecited by 28
  11. 2021Are Multilingual Models the Best Choice for Moderately Under-resourced Languages? A Comprehensive Assessment for Catalancited by 26
  12. 2021LMMS Reloaded: Transformer-based Sense Embeddings for Disambiguation and Beyondcited by 17
  13. 2019word2word: A Collection of Bilingual Lexicons for 3,564 Language Pairscited by 7
  14. 2021Bertinho: Galician BERT Representationscited by 7

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Thank You, Stingray: Multilingual Large Language Models Can Not (Yet) Disambiguate Cross-Lingual Word Sensecited by 4
  2. 2024Why We Build Local Large Language Models: An Observational Analysis from 35 Japanese and Multilingual LLMscited by 4
  3. 2025Extracting General-use Transformers for Low-resource Languages via Knowledge Distillationcited by 1
  4. 2025Multilinguality Does not Make Sense: Investigating Factors Behind Zero-Shot Transfer in Sense-Aware Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Translate to Disambiguate: Zero-shot Multilingual Word Sense Disambiguation with Pretrained Language Models
  2. 2026Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?
  3. 2024Understanding Cross-Lingual Alignment -- A Survey
  4. 2026Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech
  5. 2024Locally Measuring Cross-lingual Lexical Alignment: A Domain and Word Level Perspective
  6. 2025CALE : Concept-Aligned Embeddings for Both Within-Lemma and Inter-Lemma Sense Differentiation
  7. 2024Align, Generate, Learn: A Novel Closed-Loop Framework for Cross-Lingual In-Context Learning
  8. 2023Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignment