# Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision

arXiv: https://arxiv.org/abs/1811.10776 · HTML: https://mlsanity.com/paper/1811.10776/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2018.

## Built on by (most cited first, 3 of 3)

- [XLM-K: Improving Cross-Lingual Language Model Pre-training with Multilingual Knowledge](https://mlsanity.com/paper/2109.12573/) (2021)
- [K-XLNet: A General Method for Combining Explicit Knowledge with Language Model Pretraining](https://mlsanity.com/paper/2104.10649/) (2021)
- [Intent-aware Multi-source Contrastive Alignment for Tag-enhanced Recommendation](https://mlsanity.com/paper/2211.06370/) (2022)

## Abstract (arXiv)

Joint representation learning of words and entities benefits many NLP tasks, but has not been well explored in cross-lingual settings. In this paper, we propose a novel method for joint representation learning of cross-lingual words and entities. It captures mutually complementary knowledge, and enables cross-lingual inferences among knowledge bases and texts. Our method does not require parallel corpora, and automatically generates comparable data via distant supervision using multi-lingual knowledge bases. We utilize two types of regularizers to align cross-lingual words and entities, and design knowledge attention and cross-lingual attention to further reduce noises. We conducted a series of experiments on three tasks: word translation, entity relatedness, and cross-lingual entity linking. The results, both qualitatively and quantitatively, demonstrate the significance of our method.