mlsanityData to 14 Oct 2026

Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs

arXiv 2305.12818 ↗First posted 22 May 2023cs.CL cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

In comparative linguistics, colexification refers to the phenomenon of a lexical form conveying two or more distinct meanings. Existing work on colexification patterns relies on annotated word lists, limiting scalability and usefulness in NLP. In contrast, we identify colexification patterns of more than 2,000 concepts across 1,335 languages directly from an unannotated parallel corpus. We then propose simple and effective methods to build multilingual graphs from the colexification patterns: ColexNet and ColexNet+. ColexNet's nodes are concepts and its edges are colexifications. In ColexNet+, concept nodes are additionally linked through intermediate nodes, each representing an ngram in one of 1,334 languages. We use ColexNet+ to train overrightarrowmboxColexNet+, high-quality multilingual embeddings that are well-suited for transfer learning. In our experiments, we first show that ColexNet achieves high recall on CLICS, a dataset of crosslingual colexifications. We then evaluate overrightarrowmboxColexNet+ on roundtrip translation, sentence retrieval and sentence classification and show that our embeddings surpass several transfer learning baselines. This demonstrates the benefits of using colexification as a source of information in multilingual NLP.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 28 of its 75 references that are in the map. First posted on arXiv 22 May 2023.

Builds on

28 of 75 references.

References · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2016node2vec: Scalable Feature Learning for Networkscited by 1,918
  7. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  8. 2020Beyond English-Centric Multilingual Machine Translationcited by 561
  9. 2017Word Translation Without Parallel Datacited by 487
  10. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  11. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
  12. 2018A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddingscited by 146
16 more
  1. 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
  2. 2019SenseBERT: Driving Some Sense into BERTcited by 73
  3. 2018From Word to Sense Embeddings: A Survey on Vector Representations of Meaningcited by 49
  4. 2016Massively Multilingual Word Embeddingscited by 26
  5. 2023Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languagescited by 23
  6. 2018Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approachcited by 22
  7. 2014Multilingual Models for Compositional Distributed Semanticscited by 18
  8. 2017ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledgecited by 15
  9. 2017Past, Present, Future: A Computational Investigation of the Typology of Tense in 1000 Languagescited by 13
  10. 2023A Crosslingual Investigation of Conceptualization in 1335 Languagescited by 11
  11. 2023Language Embeddings Sometimes Contain Typological Generalizationscited by 9
  12. 2015On the universal structure of human lexical semanticscited by 8
  13. 2014An Autoencoder Approach to Learning Bilingual Word Representationscited by 7
  14. 2016A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignmentscited by 6
  15. 2017Watset: Automatic Induction of Synsets from a Graph of Synonymscited by 5
  16. 2022CaMEL: Case Marker Extraction without Labelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 2; 2026: 1.

11 papers.

  1. 2023OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretrainingcited by 37
  2. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 16
  3. 2023Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translationcited by 9
  4. 2024Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts?cited by 4
  5. 2025HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initializationcited by 3
  6. 2023Colexifications for Bootstrapping Cross-lingual Datasets: The Case of Phonology, Concreteness, and Affectivenesscited by 2
  7. 2024MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfercited by 2
  8. 2024ANGOFA: Leveraging OFA Embedding Initialization and Synthetic Data for Angolan Language Modelcited by 2
  9. 2024Patterns of Persistence and Diffusibility across the World's Languagescited by 0
  10. 2024Assessing the Role of Lexical Semantics in Cross-lingual Transfer through Controlled Manipulationscited by 0
  11. 2026BhashaSetu: Cross-Lingual Knowledge Transfer from High-Resource to Extreme Low-Resource Languagescited by 0