# Towards the generation of hierarchical attack models from cybersecurity vulnerabilities using language models

arXiv: https://arxiv.org/abs/2410.05351 · HTML: https://mlsanity.com/paper/2410.05351/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 7 Oct 2024.

## Builds on (most cited first, 9 of 9)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Deep Graph Generators: A Survey](https://mlsanity.com/paper/2012.15544/) (2020)
- [V2W-BERT: A Framework for Effective Hierarchical Multiclass Classification of Software Vulnerabilities](https://mlsanity.com/paper/2102.11498/) (2021)
- [Linking Common Vulnerabilities and Exposures to the MITRE ATT&CK Framework: A Self-Distillation Approach](https://mlsanity.com/paper/2108.01696/) (2021)
- [Generating Informative CVE Description From ExploitDB Posts by Extractive Summarization](https://mlsanity.com/paper/2101.01431/) (2021)
- [DAG-Based Attack and Defense Modeling: Don't Miss the Forest for the Attack Trees](https://mlsanity.com/paper/1303.7397/) (2013)
- [Streamlining Attack Tree Generation: A Fragment-Based Approach](https://mlsanity.com/paper/2310.00654/) (2023)

## Built on by (most cited first, 1 of 1)

- [To Patch or Not to Patch: Motivations, Challenges, and Implications for Cybersecurity](https://mlsanity.com/paper/2502.17703/) (2025)

## Abstract (arXiv)

This paper investigates the use of a pre-trained language model and siamese network to discern sibling relationships between text-based cybersecurity vulnerability data. The ultimate purpose of the approach presented in this paper is towards the construction of hierarchical attack models based on a set of text descriptions characterising potential/observed vulnerabilities in a given system. Due to the nature of the data, and the uncertainty sensitive environment in which the problem is presented, a practically oriented soft computing approach is necessary. Therefore, a key focus of this work is to investigate practical questions surrounding the reliability of predicted links towards the construction of such models, to which end conceptual and practical challenges and solutions associated with the proposed approach are outlined, such as dataset complexity and stability of predictions. Accordingly, the contributions of this paper focus on producing neural networks using a pre-trained language model for predicting sibling relationships between cybersecurity vulnerabilities, then outlining how to apply this capability towards the generation of hierarchical attack models. In addition, two data sampling mechanisms for tackling data complexity, and a consensus mechanism for reducing the amount of false positive predictions are outlined. Each of these approaches is compared and contrasted using empirical results from three sets of cybersecurity data to determine their effectiveness.