# V2W-BERT: A Framework for Effective Hierarchical Multiclass Classification of Software Vulnerabilities

arXiv: https://arxiv.org/abs/2102.11498 · HTML: https://mlsanity.com/paper/2102.11498/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 6 of 6)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [How to Fine-Tune BERT for Text Classification?](https://mlsanity.com/paper/1905.05583/) (2019)
- [TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval](https://mlsanity.com/paper/2002.06275/) (2020)

## Built on by (most cited first, 13 of 13)

- [SecureBERT: A Domain-Specific Language Model for Cybersecurity](https://mlsanity.com/paper/2204.02685/) (2022)
- [SecQA: A Concise Question-Answering Dataset for Evaluating Large Language Models in Computer Security](https://mlsanity.com/paper/2312.15838/) (2023)
- [A Survey on Data-driven Software Vulnerability Assessment and Prioritization](https://mlsanity.com/paper/2107.08364/) (2021)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [AI for DevSecOps: A Landscape and Future Opportunities](https://mlsanity.com/paper/2404.04839/) (2024)
- [CPE-Identifier: Automated CPE identification and CVE summaries annotation with Deep Learning and NLP](https://mlsanity.com/paper/2405.13568/) (2024)
- [Towards the generation of hierarchical attack models from cybersecurity vulnerabilities using language models](https://mlsanity.com/paper/2410.05351/) (2024)
- [OutCenTR: A novel semi-supervised framework for predicting exploits of vulnerabilities in high-dimensional datasets](https://mlsanity.com/paper/2304.10511/) (2023)
- [Unveiling Hidden Links Between Unseen Security Entities](https://mlsanity.com/paper/2403.02014/) (2024)
- [CTIGuardian: A Few-Shot Framework for Mitigating Privacy Leakage in Fine-Tuned LLMs](https://mlsanity.com/paper/2512.12914/) (2025)
- [Fine-tuning RoBERTa for CVE-to-CWE Classification: A 125M Parameter Model Competitive with LLMs](https://mlsanity.com/paper/2603.14911/) (2026)
- [From CVE to CWE: Syscall-Based HIDS Generalisation](https://mlsanity.com/paper/2606.22581/) (2026)
- [Mitigating The Effect of Class Imbalance in Data with Hierarchical and Dependable Structure](https://mlsanity.com/paper/2607.11994/) (2026)

## Abstract (arXiv)

Weaknesses in computer systems such as faults, bugs and errors in the architecture, design or implementation of software provide vulnerabilities that can be exploited by attackers to compromise the security of a system. Common Weakness Enumerations (CWE) are a hierarchically designed dictionary of software weaknesses that provide a means to understand software flaws, potential impact of their exploitation, and means to mitigate these flaws. Common Vulnerabilities and Exposures (CVE) are brief low-level descriptions that uniquely identify vulnerabilities in a specific product or protocol. Classifying or mapping of CVEs to CWEs provides a means to understand the impact and mitigate the vulnerabilities. Since manual mapping of CVEs is not a viable option, automated approaches are desirable but challenging. We present a novel Transformer-based learning framework (V2W-BERT) in this paper. By using ideas from natural language processing, link prediction and transfer learning, our method outperforms previous approaches not only for CWE instances with abundant data to train, but also rare CWE classes with little or no data to train. Our approach also shows significant improvements in using historical data to predict links for future instances of CVEs, and therefore, provides a viable approach for practical applications. Using data from MITRE and National Vulnerability Database, we achieve up to 97% prediction accuracy for randomly partitioned data and up to 94% prediction accuracy in temporally partitioned data. We believe that our work will influence the design of better methods and training models, as well as applications to solve increasingly harder problems in cybersecurity.