# CVSS-BERT: Explainable Natural Language Processing to Determine the Severity of a Computer Security Vulnerability from its Description

arXiv: https://arxiv.org/abs/2111.08510 · HTML: https://mlsanity.com/paper/2111.08510/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 16 Nov 2021.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)
- [The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?](https://mlsanity.com/paper/2010.05607/) (2020)
- [Exploit Prediction Scoring System (EPSS)](https://mlsanity.com/paper/1908.04856/) (2019)
- [Analyzing the Perceived Severity of Cybersecurity Threats Reported on Social Media](https://mlsanity.com/paper/1902.10680/) (2019)

## Built on by (most cited first, 10 of 10)

- [SecQA: A Concise Question-Answering Dataset for Evaluating Large Language Models in Computer Security](https://mlsanity.com/paper/2312.15838/) (2023)
- [On the Uses of Large Language Models to Interpret Ambiguous Cyberattack Descriptions](https://mlsanity.com/paper/2306.14062/) (2023)
- [Crimson: Empowering Strategic Reasoning in Cybersecurity through Large Language Models](https://mlsanity.com/paper/2403.00878/) (2024)
- [CVE-LLM : Ontology-Assisted Automatic Vulnerability Evaluation Using Large Language Models](https://mlsanity.com/paper/2502.15932/) (2025)
- [Exploring the Limits of Transfer Learning with Unified Model in the Cybersecurity Domain](https://mlsanity.com/paper/2302.10346/) (2023)
- [CPE-Identifier: Automated CPE identification and CVE summaries annotation with Deep Learning and NLP](https://mlsanity.com/paper/2405.13568/) (2024)
- [Common Vulnerability Scoring System Prediction based on Open Source Intelligence Information Sources](https://mlsanity.com/paper/2210.02143/) (2022)
- [Enhancing Vulnerability Reports with Automated and Augmented Description Summarization](https://mlsanity.com/paper/2504.20726/) (2025)
- [CVE-LLM : Automatic vulnerability evaluation in medical device industry using large language models](https://mlsanity.com/paper/2407.14640/) (2024)
- [A Modality-Aware Cooperative Co-Evolutionary Framework for Multimodal Graph Neural Architecture Search](https://mlsanity.com/paper/2510.07325/) (2025)

## Abstract (arXiv)

When a new computer security vulnerability is publicly disclosed, only a textual description of it is available. Cybersecurity experts later provide an analysis of the severity of the vulnerability using the Common Vulnerability Scoring System (CVSS). Specifically, the different characteristics of the vulnerability are summarized into a vector (consisting of a set of metrics), from which a severity score is computed. However, because of the high number of vulnerabilities disclosed everyday this process requires lot of manpower, and several days may pass before a vulnerability is analyzed. We propose to leverage recent advances in the field of Natural Language Processing (NLP) to determine the CVSS vector and the associated severity score of a vulnerability from its textual description in an explainable manner. To this purpose, we trained multiple BERT classifiers, one for each metric composing the CVSS vector. Experimental results show that our trained classifiers are able to determine the value of the metrics of the CVSS vector with high accuracy. The severity score computed from the predicted CVSS vector is also very close to the real severity score attributed by a human expert. For explainability purpose, gradient-based input saliency method was used to determine the most relevant input words for a given prediction made by our classifiers. Often, the top relevant words include terms in agreement with the rationales of a human cybersecurity expert, making the explanation comprehensible for end-users.