# Crimson: Empowering Strategic Reasoning in Cybersecurity through Large Language Models

arXiv: https://arxiv.org/abs/2403.00878 · HTML: https://mlsanity.com/paper/2403.00878/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 13 of its 16 references that are in the map.
- First posted on arXiv 1 Mar 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 13 of 13)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022)
- [Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine](https://mlsanity.com/paper/2311.16452/) (2023)
- [Task-aware Retrieval with Instructions](https://mlsanity.com/paper/2211.09260/) (2022)
- [AttacKG: Constructing Technique Knowledge Graph from Cyber Threat Intelligence Reports](https://mlsanity.com/paper/2111.07093/) (2021)
- [Looking Beyond IoCs: Automatically Extracting Attack Patterns from External CTI](https://mlsanity.com/paper/2211.01753/) (2022)
- [CVSS-BERT: Explainable Natural Language Processing to Determine the Severity of a Computer Security Vulnerability from its Description](https://mlsanity.com/paper/2111.08510/) (2021)
- [ThreatKG: An AI-Powered System for Automated Open-Source Cyber Threat Intelligence Gathering and Management](https://mlsanity.com/paper/2212.10388/) (2022)
- [Vox Populi, Vox ChatGPT: Large Language Models, Education and Democracy](https://mlsanity.com/paper/2311.06207/) (2023)

## Built on by (most cited first, 8 of 8)

- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Audit-LLM: Multi-Agent Collaboration for Log-based Insider Threat Detection](https://mlsanity.com/paper/2408.08902/) (2024)
- [ExCyTIn-Bench: Evaluating LLM agents on Cyber Threat Investigation](https://mlsanity.com/paper/2507.14201/) (2025)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [Ollabench: Evaluating LLMs' Reasoning for Human-centric Interdependent Cybersecurity](https://mlsanity.com/paper/2406.06863/) (2024)
- [Automated Post-Incident Policy Gap Analysis via Threat-Informed Evidence Mapping using Large Language Models](https://mlsanity.com/paper/2601.03287/) (2026)
- [On Automating Security Policies with Contemporary LLMs](https://mlsanity.com/paper/2506.04838/) (2025)
- [Reasoning Topology Matters: A Controlled Study of LLM-Based Cybersecurity Analysis](https://mlsanity.com/paper/2609.24710/) (2026)

## Abstract (arXiv)

We introduces Crimson, a system that enhances the strategic reasoning capabilities of Large Language Models (LLMs) within the realm of cybersecurity. By correlating CVEs with MITRE ATT&CK techniques, Crimson advances threat anticipation and strategic defense efforts. Our approach includes defining and evaluating cybersecurity strategic tasks, alongside implementing a comprehensive human-in-the-loop data-synthetic workflow to develop the CVE-to-ATT&CK Mapping (CVEM) dataset. We further enhance LLMs' reasoning abilities through a novel Retrieval-Aware Training (RAT) process and its refined iteration, RAT-R. Our findings demonstrate that an LLM fine-tuned with our techniques, possessing 7 billion parameters, approaches the performance level of GPT-4, showing markedly lower rates of hallucination and errors, and surpassing other models in strategic reasoning tasks. Moreover, domain-specific fine-tuning of embedding models significantly improves performance within cybersecurity contexts, underscoring the efficacy of our methodology. By leveraging Crimson to convert raw vulnerability data into structured and actionable insights, we bolster proactive cybersecurity defenses.