# Learning Domain-Specific Word Embeddings from Sparse Cybersecurity Texts

arXiv: https://arxiv.org/abs/1709.07470 · HTML: https://mlsanity.com/paper/1709.07470/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2017.

## Built on by (most cited first, 3 of 3)

- [T-VEC: A Telecom-Specific Vectorization Model with Enhanced Semantic Understanding via Deep Triplet Loss Fine-Tuning](https://mlsanity.com/paper/2504.16460/) (2025)
- [Evaluation of Word Embeddings for the Social Sciences](https://mlsanity.com/paper/2302.06174/) (2023)
- [DNS-GT: A Graph-based Transformer Approach to Learn Embeddings of Domain Names from DNS Queries](https://mlsanity.com/paper/2603.11200/) (2026)

## Abstract (arXiv)

Word embedding is a Natural Language Processing (NLP) technique that automatically maps words from a vocabulary to vectors of real numbers in an embedding space. It has been widely used in recent years to boost the performance of a vari-ety of NLP tasks such as Named Entity Recognition, Syntac-tic Parsing and Sentiment Analysis. Classic word embedding methods such as Word2Vec and GloVe work well when they are given a large text corpus. When the input texts are sparse as in many specialized domains (e.g., cybersecurity), these methods often fail to produce high-quality vectors. In this pa-per, we describe a novel method to train domain-specificword embeddings from sparse texts. In addition to domain texts, our method also leverages diverse types of domain knowledge such as domain vocabulary and semantic relations. Specifi-cally, we first propose a general framework to encode diverse types of domain knowledge as text annotations. Then we de-velop a novel Word Annotation Embedding (WAE) algorithm to incorporate diverse types of text annotations in word em-bedding. We have evaluated our method on two cybersecurity text corpora: a malware description corpus and a Common Vulnerability and Exposure (CVE) corpus. Our evaluation re-sults have demonstrated the effectiveness of our method in learning domain-specific word embeddings.