# A Comparison of Word2Vec, HMM2Vec, and PCA2Vec for Malware Classification

arXiv: https://arxiv.org/abs/2103.05763 · HTML: https://mlsanity.com/paper/2103.05763/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 50 references that are in the map.
- First posted on arXiv 7 Mar 2021.

## Builds on (most cited first, 3 of 3)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [A Tutorial on Principal Component Analysis](https://mlsanity.com/paper/1404.1100/) (2014)

## Built on by (most cited first, 4 of 4)

- [Generative Artificial Intelligence: A Systematic Review and Applications](https://mlsanity.com/paper/2405.11029/) (2024)
- [A Comprehensive Study on Learning-Based PE Malware Family Classification Methods](https://mlsanity.com/paper/2110.15552/) (2021)
- [Hidden Markov Models with Momentum](https://mlsanity.com/paper/2206.04057/) (2022)
- [Classifying World War II Era Ciphers with Machine Learning](https://mlsanity.com/paper/2307.00501/) (2023)

## Abstract (arXiv)

Word embeddings are often used in natural language processing as a means to quantify relationships between words. More generally, these same word embedding techniques can be used to quantify relationships between features. In this paper, we first consider multiple different word embedding techniques within the context of malware classification. We use hidden Markov models to obtain embedding vectors in an approach that we refer to as HMM2Vec, and we generate vector embeddings based on principal component analysis. We also consider the popular neural network based word embedding technique known as Word2Vec. In each case, we derive feature embeddings based on opcode sequences for malware samples from a variety of different families. We show that we can obtain better classification accuracy based on these feature embeddings, as compared to HMM experiments that directly use the opcode sequences, and serve to establish a baseline. These results show that word embeddings can be a useful feature engineering step in the field of malware analysis.