# Measuring Word Significance using Distributed Representations of Words

arXiv: https://arxiv.org/abs/1508.02297 · HTML: https://mlsanity.com/paper/1508.02297/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2015.

## Built on by (most cited first, 21 of 21)

- [CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions](https://mlsanity.com/paper/2112.05219/) (2021)
- [Incorporating Residual and Normalization Layers into Analysis of Masked Language Models](https://mlsanity.com/paper/2109.07152/) (2021)
- [A Simple Hash-Based Early Exiting Approach For Language Understanding and Generation](https://mlsanity.com/paper/2203.01670/) (2022)
- [Variational Graph Normalized Auto-Encoders](https://mlsanity.com/paper/2108.08046/) (2021)
- [CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation](https://mlsanity.com/paper/2305.14635/) (2023)
- [Norm of Word Embedding Encodes Information Gain](https://mlsanity.com/paper/2212.09663/) (2022)
- [Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings](https://mlsanity.com/paper/2406.10984/) (2024)
- [Revisiting Additive Compositionality: AND, OR and NOT Operations with Word Embeddings](https://mlsanity.com/paper/2105.08585/) (2021)
- [Improving Zero-Shot Multilingual Translation with Universal Representations and Cross-Mappings](https://mlsanity.com/paper/2210.15851/) (2022)
- [Importance-Aware Data Augmentation for Document-Level Neural Machine Translation](https://mlsanity.com/paper/2401.15360/) (2024)
- [Monitoring geometrical properties of word embeddings for detecting the emergence of new topics](https://mlsanity.com/paper/2111.03496/) (2021)
- [Probing with Noise: Unpicking the Warp and Weft of Embeddings](https://mlsanity.com/paper/2210.12206/) (2022)
- [Zipfian Whitening](https://mlsanity.com/paper/2411.00680/) (2024)
- [Assessing Word Importance Using Models Trained for Semantic Tasks](https://mlsanity.com/paper/2305.19689/) (2023)
- [Language on Demand, Knowledge at Core: Composing LLMs with Encoder-Decoder Translation Models for Extensible Multilinguality](https://mlsanity.com/paper/2603.17512/) (2026)
- [Norm of Mean Contextualized Embeddings Determines their Variance](https://mlsanity.com/paper/2409.11253/) (2024)
- [Geometry-Guided Adversarial Prompt Detection via Curvature and Local Intrinsic Dimension](https://mlsanity.com/paper/2503.03502/) (2025)
- [Learning Word Embedding with Better Distance Weighting and Window Size Scheduling](https://mlsanity.com/paper/2404.14631/) (2024)
- [In Tree Structure Should Sentence Be Generated](https://mlsanity.com/paper/2406.14189/) (2024)
- [LIDS: LLM Summary Inference Under the Layered Lens](https://mlsanity.com/paper/2603.00105/) (2026)
- [N-GRPO: Embedding-Level Neighbor Mixing for Enhanced Policy Optimization](https://mlsanity.com/paper/2606.10768/) (2026)

## Abstract (arXiv)

Distributed representations of words as real-valued vectors in a relatively low-dimensional space aim at extracting syntactic and semantic features from large text corpora. A recently introduced neural network, named word2vec (Mikolov et al., 2013a; Mikolov et al., 2013b), was shown to encode semantic information in the direction of the word vectors. In this brief report, it is proposed to use the length of the vectors, together with the term frequency, as measure of word significance in a corpus. Experimental evidence using a domain-specific corpus of abstracts is presented to support this proposal. A useful visualization technique for text corpora emerges, where words are mapped onto a two-dimensional plane and automatically ranked by significance.