# Continuous multilinguality with language vectors

arXiv: https://arxiv.org/abs/1612.07486 · HTML: https://mlsanity.com/paper/1612.07486/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2016.

## Built on by (most cited first, 20 of 20)

- [Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference](https://mlsanity.com/paper/2110.03742/) (2021)
- [A Survey of Multilingual Reasoning in Language Models](https://mlsanity.com/paper/2502.09457/) (2025)
- [Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?](https://mlsanity.com/paper/2409.19151/) (2024)
- [Does Typological Blinding Impede Cross-Lingual Sharing?](https://mlsanity.com/paper/2101.11888/) (2021)
- [Language Embeddings Sometimes Contain Typological Generalizations](https://mlsanity.com/paper/2301.08115/) (2023)
- [Multilingual Gradient Word-Order Typology from Universal Dependencies](https://mlsanity.com/paper/2402.01513/) (2024)
- [Language Embeddings for Typology and Cross-lingual Transfer Learning](https://mlsanity.com/paper/2106.02082/) (2021)
- [Large Language Models are Easily Confused: A Quantitative Metric, Security Implications and Typological Analysis](https://mlsanity.com/paper/2410.13237/) (2024)
- [Measuring Geographic Performance Disparities of Offensive Language Classifiers](https://mlsanity.com/paper/2209.07353/) (2022)
- [Discovering Representation Sprachbund For Multilingual Pre-Training](https://mlsanity.com/paper/2109.00271/) (2021)
- [On the Universality of Deep Contextual Language Models](https://mlsanity.com/paper/2109.07140/) (2021)
- [Language Models Are Implicitly Continuous](https://mlsanity.com/paper/2504.03933/) (2025)
- [Towards Zero-shot Language Modeling](https://mlsanity.com/paper/2108.03334/) (2021)
- [Neighbors and relatives: How do speech embeddings reflect linguistic connections across the world?](https://mlsanity.com/paper/2506.08564/) (2025)
- [LangSAMP: Language-Script Aware Multilingual Pretraining](https://mlsanity.com/paper/2409.18199/) (2024)
- [Informative Language Representation Learning for Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/2209.01530/) (2022)
- [We went to look for meaning and all we got were these lousy representations: aspects of meaning representation for computational semantics](https://mlsanity.com/paper/2109.04949/) (2021)
- [mPLM-Sim: Better Cross-Lingual Similarity and Transfer in Multilingual Pretrained Language Models](https://mlsanity.com/paper/2305.13684/) (2023)
- [MaiNLP at SemEval-2024 Task 1: Analyzing Source Language Selection in Cross-Lingual Textual Relatedness](https://mlsanity.com/paper/2404.02570/) (2024)
- [Self-Supervised Lexical Representation Learning for Fast, Large-Scale Phylogenetic Inference](https://mlsanity.com/paper/2609.05262/) (2026)

## Abstract (arXiv)

Most existing models for multilingual natural language processing (NLP) treat language as a discrete category, and make predictions for either one language or the other. In contrast, we propose using continuous vector representations of language. We show that these can be learned efficiently with a character-based neural language model, and used to improve inference about language varieties not seen during training. In experiments with 1303 Bible translations into 990 different languages, we empirically explore the capacity of multilingual language models, and also show that the language vectors capture genetic relationships between languages.