# Crosslingual Transfer Learning for Low-Resource Languages Based on Multilingual Colexification Graphs

arXiv: https://arxiv.org/abs/2305.12818 · HTML: https://mlsanity.com/paper/2305.12818/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 75 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 28 of 28)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://mlsanity.com/paper/1805.06297/) (2018)
- [Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages](https://mlsanity.com/paper/2305.12182/) (2023)
- [SenseBERT: Driving Some Sense into BERT](https://mlsanity.com/paper/1908.05646/) (2019)
- [From Word to Sense Embeddings: A Survey on Vector Representations of Meaning](https://mlsanity.com/paper/1805.04032/) (2018)
- [Massively Multilingual Word Embeddings](https://mlsanity.com/paper/1602.01925/) (2016)
- [Taxi1500: A Multilingual Dataset for Text Classification in 1500 Languages](https://mlsanity.com/paper/2305.08487/) (2023)
- [Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approach](https://mlsanity.com/paper/1808.08773/) (2018)
- [Multilingual Models for Compositional Distributed Semantics](https://mlsanity.com/paper/1404.4641/) (2014)
- [ConceptNet at SemEval-2017 Task 2: Extending Word Embeddings with Multilingual Relational Knowledge](https://mlsanity.com/paper/1704.03560/) (2017)
- [Past, Present, Future: A Computational Investigation of the Typology of Tense in 1000 Languages](https://mlsanity.com/paper/1704.08914/) (2017)
- [A Crosslingual Investigation of Conceptualization in 1335 Languages](https://mlsanity.com/paper/2305.08475/) (2023)
- [Language Embeddings Sometimes Contain Typological Generalizations](https://mlsanity.com/paper/2301.08115/) (2023)
- [On the universal structure of human lexical semantics](https://mlsanity.com/paper/1504.07843/) (2015)
- [An Autoencoder Approach to Learning Bilingual Word Representations](https://mlsanity.com/paper/1402.1454/) (2014)
- [A Strong Baseline for Learning Cross-Lingual Word Embeddings from Sentence Alignments](https://mlsanity.com/paper/1608.05426/) (2016)
- [Watset: Automatic Induction of Synsets from a Graph of Synonyms](https://mlsanity.com/paper/1704.07157/) (2017)
- [CaMEL: Case Marker Extraction without Labels](https://mlsanity.com/paper/2203.10010/) (2022)

## Built on by (most cited first, 11 of 11)

- [OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining](https://mlsanity.com/paper/2311.08849/) (2023)
- [Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead](https://mlsanity.com/paper/2505.21315/) (2025)
- [Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation](https://mlsanity.com/paper/2305.14189/) (2023)
- [Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts?](https://mlsanity.com/paper/2411.04530/) (2024)
- [HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization](https://mlsanity.com/paper/2504.21018/) (2025)
- [Colexifications for Bootstrapping Cross-lingual Datasets: The Case of Phonology, Concreteness, and Affectiveness](https://mlsanity.com/paper/2306.02646/) (2023)
- [MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer](https://mlsanity.com/paper/2401.04821/) (2024)
- [ANGOFA: Leveraging OFA Embedding Initialization and Synthetic Data for Angolan Language Model](https://mlsanity.com/paper/2404.02534/) (2024)
- [Patterns of Persistence and Diffusibility across the World's Languages](https://mlsanity.com/paper/2401.01698/) (2024)
- [Assessing the Role of Lexical Semantics in Cross-lingual Transfer through Controlled Manipulations](https://mlsanity.com/paper/2408.07599/) (2024)
- [BhashaSetu: Cross-Lingual Knowledge Transfer from High-Resource to Extreme Low-Resource Languages](https://mlsanity.com/paper/2602.05599/) (2026)

## Abstract (arXiv)

In comparative linguistics, colexification refers to the phenomenon of a lexical form conveying two or more distinct meanings. Existing work on colexification patterns relies on annotated word lists, limiting scalability and usefulness in NLP. In contrast, we identify colexification patterns of more than 2,000 concepts across 1,335 languages directly from an unannotated parallel corpus. We then propose simple and effective methods to build multilingual graphs from the colexification patterns: ColexNet and ColexNet+. ColexNet's nodes are concepts and its edges are colexifications. In ColexNet+, concept nodes are additionally linked through intermediate nodes, each representing an ngram in one of 1,334 languages. We use ColexNet+ to train overrightarrowmboxColexNet+, high-quality multilingual embeddings that are well-suited for transfer learning. In our experiments, we first show that ColexNet achieves high recall on CLICS, a dataset of crosslingual colexifications. We then evaluate overrightarrowmboxColexNet+ on roundtrip translation, sentence retrieval and sentence classification and show that our embeddings surpass several transfer learning baselines. This demonstrates the benefits of using colexification as a source of information in multilingual NLP.