# Leveraging knowledge graphs to update scientific word embeddings using latent semantic imputation

arXiv: https://arxiv.org/abs/2210.15358 · HTML: https://mlsanity.com/paper/2210.15358/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 52 references that are in the map.
- First posted on arXiv 27 Oct 2022.

## Builds on (most cited first, 25 of 25)

- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets](https://mlsanity.com/paper/1906.05474/) (2019)
- [SciFive: a text-to-text transformer model for biomedical literature](https://mlsanity.com/paper/2106.03598/) (2021)
- [Mimicking Word Embeddings using Subword RNNs](https://mlsanity.com/paper/1707.06961/) (2017)
- [A Comparison of Word Embeddings for the Biomedical Natural Language Processing](https://mlsanity.com/paper/1802.00400/) (2018)
- [Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation](https://mlsanity.com/paper/1508.02096/) (2015)
- [Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approach](https://mlsanity.com/paper/1808.08773/) (2018)
- [Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models](https://mlsanity.com/paper/1604.00788/) (2016)
- [Unsupervised Hyperalignment for Multilingual Word Embeddings](https://mlsanity.com/paper/1811.01124/) (2018)
- [SECNLP: A Survey of Embeddings in Clinical Natural Language Processing](https://mlsanity.com/paper/1903.01039/) (2019)
- [Compositional Morphology for Word Representations and Language Modelling](https://mlsanity.com/paper/1405.4273/) (2014)
- [How to Generate a Good Word Embedding?](https://mlsanity.com/paper/1507.05523/) (2015)
- [Generalizing Word Embeddings using Bag of Subwords](https://mlsanity.com/paper/1809.04259/) (2018)
- [Beyond Word Embeddings: Learning Entity and Concept Representations from Large Scale Knowledge Bases](https://mlsanity.com/paper/1801.00388/) (2018)
- [Characterizing Diseases from Unstructured Text: A Vocabulary Driven Word2vec Approach](https://mlsanity.com/paper/1603.00106/) (2016)
- [Extrofitting: Enriching Word Representation and its Vector Space with Semantic Lexicons](https://mlsanity.com/paper/1804.07946/) (2018)
- [A Simple Regularization-based Algorithm for Learning Cross-Domain Word Embeddings](https://mlsanity.com/paper/1902.00184/) (2019)
- [Enhancing Domain Word Embedding via Latent Semantic Imputation](https://mlsanity.com/paper/1905.08900/) (2019)
- [Can Embeddings Adequately Represent Medical Terminology? New Large-Scale Medical Term Similarity Datasets Have the Answer!](https://mlsanity.com/paper/2003.11082/) (2020)
- [Retrofitting Concept Vector Representations of Medical Concepts to Improve Estimates of Semantic Similarity and Relatedness](https://mlsanity.com/paper/1709.07357/) (2017)
- [Utility of General and Specific Word Embeddings for Classifying Translational Stages of Research](https://mlsanity.com/paper/1705.06262/) (2017)

## Built on by (most cited first, 1 of 1)

- [The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers](https://mlsanity.com/paper/2506.20844/) (2025)

## Abstract (arXiv)

The most interesting words in scientific texts will often be novel or rare. This presents a challenge for scientific word embedding models to determine quality embedding vectors for useful terms that are infrequent or newly emerging. We demonstrate how glslsi can address this problem by imputing embeddings for domain-specific words from up-to-date knowledge graphs while otherwise preserving the original word embedding model. We use the MeSH knowledge graph to impute embedding vectors for biomedical terminology without retraining and evaluate the resulting embedding model on a domain-specific word-pair similarity task. We show that LSI can produce reliable embedding vectors for rare and OOV terms in the biomedical domain.