# From Description to Score: Can LLMs Quantify Vulnerabilities?

arXiv: https://arxiv.org/abs/2512.06781 · HTML: https://mlsanity.com/paper/2512.06781/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 30 references that are in the map.
- First posted on arXiv 7 Dec 2025.

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models](https://mlsanity.com/paper/2410.05229/) (2024)
- [LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks](https://mlsanity.com/paper/2312.12575/) (2023)
- [CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligence](https://mlsanity.com/paper/2406.07599/) (2024)
- [Pushing The Limit of LLM Capacity for Text Classification](https://mlsanity.com/paper/2402.07470/) (2024)
- [Can LLMs Classify CVEs? Investigating LLMs Capabilities in Computing CVSS Vectors](https://mlsanity.com/paper/2504.10713/) (2025)
- [Shedding Light on CVSS Scoring Inconsistencies: A User-Centric Study on Evaluating Widespread Security Vulnerabilities](https://mlsanity.com/paper/2308.15259/) (2023)

## Built on by (most cited first, 1 of 1)

- [Contextualization of Third-Party Cloud Security Findings](https://mlsanity.com/paper/2610.08895/) (2026)

## Abstract (arXiv)

Manual vulnerability scoring, such as assigning Common Vulnerability Scoring System (CVSS) scores, is a resource-intensive process that is often influenced by subjective interpretation. This study investigates the potential of general-purpose large language models (LLMs), namely ChatGPT, Llama, Grok, DeepSeek, and Gemini, to automate this process by analyzing over 31,000 recent Common Vulnerabilities and Exposures (CVE) entries. The results show that LLMs substantially outperform the baseline on certain metrics (e.g., textitAvailability Impact), while offering more modest gains on others (e.g., textitAttack Complexity). Moreover, model performance varies across both LLM families and individual CVSS metrics, with ChatGPT-5 attaining the highest precision. Our analysis reveals that LLMs tend to misclassify many of the same CVEs, and ensemble-based meta-classifiers only marginally improve performance. Further examination shows that CVE descriptions often lack critical context or contain ambiguous phrasing, which contributes to systematic misclassifications. These findings underscore the importance of enhancing vulnerability descriptions and incorporating richer contextual details to support more reliable automated reasoning and alleviate the growing backlog of CVEs awaiting triage.