# The Application of Transformer-Based Models for Predicting Consequences of Cyber Attacks

arXiv: https://arxiv.org/abs/2508.13030 · HTML: https://mlsanity.com/paper/2508.13030/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 45 references that are in the map.
- First posted on arXiv 18 Aug 2025.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective](https://mlsanity.com/paper/2209.13232/) (2022)
- [LSTM-Autoencoder based Anomaly Detection for Indoor Air Quality Time Series Data](https://mlsanity.com/paper/2204.06701/) (2022)
- [Detection of Cyberbullying Incidents on the Instagram Social Network](https://mlsanity.com/paper/1503.03909/) (2015)
- [Transformer-based Generative Adversarial Networks in Computer Vision: A Comprehensive Survey](https://mlsanity.com/paper/2302.08641/) (2023)
- [Survey of Malware Analysis through Control Flow Graph using Machine Learning](https://mlsanity.com/paper/2305.08993/) (2023)
- [DomURLs_BERT: Pre-trained BERT-based Model for Malicious Domains and URLs Detection and Classification](https://mlsanity.com/paper/2409.09143/) (2024)
- [Flow-based Network Intrusion Detection Based on BERT Masked Language Model](https://mlsanity.com/paper/2306.04920/) (2023)
- [LSTM Recurrent Neural Networks for Cybersecurity Named Entity Recognition](https://mlsanity.com/paper/2409.10521/) (2024)

## Built on by (most cited first, 1 of 1)

- [Mitigating The Effect of Class Imbalance in Data with Hierarchical and Dependable Structure](https://mlsanity.com/paper/2607.11994/) (2026)

## Abstract (arXiv)

Cyberattacks are increasing, and securing against such threats is costing industries billions of dollars annually. Threat Modeling, that is, comprehending the consequences of these attacks, can provide critical support to cybersecurity professionals, enabling them to take timely action and allocate resources that could be used elsewhere. Cybersecurity is heavily dependent on threat modeling, as it assists security experts in assessing and mitigating risks related to identifying vulnerabilities and threats. Recently, there has been a pressing need for automated methods to assess attack descriptions and forecast the future consequences of the increasing complexity of cyberattacks. This study examines how Natural Language Processing (NLP) and deep learning can be applied to analyze the potential impact of cyberattacks by leveraging textual descriptions from the MITRE Common Weakness Enumeration (CWE) database. We emphasize classifying attack consequences into five principal categories: Availability, Access Control, Confidentiality, Integrity, and Other. This paper investigates the use of Bidirectional Encoder Representations from Transformers (BERT) in combination with Hierarchical Attention Networks (HANs) for Multi-label classification, evaluating their performance in comparison with conventional CNN and LSTM-based models. Experimental findings show that BERT achieves an overall accuracy of 0.972, far higher than conventional deep learning models in multi-label classification. HAN outperforms baseline forms of CNN and LSTM-based models on specific cybersecurity labels. However, BERT consistently achieves better precision and recall, making it more suitable for predicting the consequences of a cyberattack.