# Uncovering divergent linguistic information in word embeddings with lessons for intrinsic and extrinsic evaluation

arXiv: https://arxiv.org/abs/1809.02094 · HTML: https://mlsanity.com/paper/1809.02094/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2018.

## Built on by (most cited first, 8 of 8)

- [DSE-GAN: Dynamic Semantic Evolution Generative Adversarial Network for Text-to-Image Generation](https://mlsanity.com/paper/2209.01339/) (2022)
- [LSCDiscovery: A shared task on semantic change discovery and detection in Spanish](https://mlsanity.com/paper/2205.06691/) (2022)
- [Using Context-to-Vector with Graph Retrofitting to Improve Word Embeddings](https://mlsanity.com/paper/2210.16848/) (2022)
- [Semantic and Expressive Variation in Image Captions Across Languages](https://mlsanity.com/paper/2310.14356/) (2023)
- [Effects of Pre- and Post-Processing on type-based Embeddings in Lexical Semantic Change Detection](https://mlsanity.com/paper/2101.09368/) (2021)
- [Data Augmentation to Address Out-of-Vocabulary Problem in Low-Resource Sinhala-English Neural Machine Translation](https://mlsanity.com/paper/2205.08722/) (2022)
- [A survey of neural-network-based methods utilising comparable data for finding translation equivalents](https://mlsanity.com/paper/2410.15144/) (2024)
- [Unsupervised Bilingual Lexicon Induction for Low Resource Languages](https://mlsanity.com/paper/2412.16894/) (2024)

## Abstract (arXiv)

Following the recent success of word embeddings, it has been argued that there is no such thing as an ideal representation for words, as different models tend to capture divergent and often mutually incompatible aspects like semantics/syntax and similarity/relatedness. In this paper, we show that each embedding model captures more information than directly apparent. A linear transformation that adjusts the similarity order of the model without any external resource can tailor it to achieve better results in those aspects, providing a new perspective on how embeddings encode divergent linguistic information. In addition, we explore the relation between intrinsic and extrinsic evaluation, as the effect of our transformations in downstream tasks is higher for unsupervised systems than for supervised ones.