# Evaluation of taxonomic and neural embedding methods for calculating semantic similarity

arXiv: https://arxiv.org/abs/2209.15197 · HTML: https://mlsanity.com/paper/2209.15197/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 127 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Assessing BERT's Syntactic Abilities](https://mlsanity.com/paper/1901.05287/) (2019)
- [SimLex-999: Evaluating Semantic Models with (Genuine) Similarity Estimation](https://mlsanity.com/paper/1408.3456/) (2014)
- [Dissecting Contextual Word Embeddings: Architecture and Representation](https://mlsanity.com/paper/1808.08949/) (2018)
- [Counter-fitting Word Vectors to Linguistic Constraints](https://mlsanity.com/paper/1603.00892/) (2016)
- [Troubling Trends in Machine Learning Scholarship](https://mlsanity.com/paper/1807.03341/) (2018)
- [Learned in Translation: Contextualized Word Vectors](https://mlsanity.com/paper/1708.00107/) (2017)
- [On the Dimensionality of Word Embedding](https://mlsanity.com/paper/1812.04224/) (2018)
- [SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity](https://mlsanity.com/paper/1608.00869/) (2016)
- [Simple and Effective Dimensionality Reduction for Word Embeddings](https://mlsanity.com/paper/1708.03629/) (2017)
- [Semantic Specialisation of Distributional Word Vector Spaces using Monolingual and Cross-Lingual Constraints](https://mlsanity.com/paper/1706.00374/) (2017)
- [An Ensemble Method to Produce High-Quality Word Embeddings (2016)](https://mlsanity.com/paper/1604.01692/) (2016)
- [Adversarial Propagation and Zero-Shot Cross-Lingual Transfer of Word Vector Specialization](https://mlsanity.com/paper/1809.04163/) (2018)
- [Card-660: Cambridge Rare Word Dataset - a Reliable Benchmark for Infrequent Word Representation Models](https://mlsanity.com/paper/1808.09308/) (2018)
- [Retrofitting Contextualized Word Embeddings with Paraphrases](https://mlsanity.com/paper/1909.09700/) (2019)
- [Semantic Similarity from Natural Language and Ontology Analysis](https://mlsanity.com/paper/1704.05295/) (2017)

## Built on by (most cited first, 2 of 2)

- [Lexical semantics enhanced neural word embeddings](https://mlsanity.com/paper/2210.00754/) (2022)
- [Advances and Challenges in Semantic Textual Similarity: A Comprehensive Survey](https://mlsanity.com/paper/2601.03270/) (2025)

## Abstract (arXiv)

Modelling semantic similarity plays a fundamental role in lexical semantic applications. A natural way of calculating semantic similarity is to access handcrafted semantic networks, but similarity prediction can also be anticipated in a distributional vector space. Similarity calculation continues to be a challenging task, even with the latest breakthroughs in deep neural language models. We first examined popular methodologies in measuring taxonomic similarity, including edge-counting that solely employs semantic relations in a taxonomy, as well as the complex methods that estimate concept specificity. We further extrapolated three weighting factors in modelling taxonomic similarity. To study the distinct mechanisms between taxonomic and distributional similarity measures, we ran head-to-head comparisons of each measure with human similarity judgements from the perspectives of word frequency, polysemy degree and similarity intensity. Our findings suggest that without fine-tuning the uniform distance, taxonomic similarity measures can depend on the shortest path length as a prime factor to predict semantic similarity; in contrast to distributional semantics, edge-counting is free from sense distribution bias in use and can measure word similarity both literally and metaphorically; the synergy of retrofitting neural embeddings with concept relations in similarity prediction may indicate a new trend to leverage knowledge bases on transfer learning. It appears that a large gap still exists on computing semantic similarity among different ranges of word frequency, polysemous degree and similarity intensity.