# Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making

arXiv: https://arxiv.org/abs/2608.20966 · HTML: https://mlsanity.com/paper/2608.20966/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Builds on (most cited first, 11 of 11)

- [LLMs Get Lost In Multi-Turn Conversation](https://mlsanity.com/paper/2505.06120/) (2025)
- [A Review on Bilevel Optimization: From Classical to Evolutionary Approaches and Applications](https://mlsanity.com/paper/1705.06270/) (2017)
- [Unleashing the potential of prompt engineering for large language models](https://mlsanity.com/paper/2310.14735/) (2023)
- [Primus: A Pioneering Collection of Open-Source Datasets for Cybersecurity LLM Training](https://mlsanity.com/paper/2502.11191/) (2025)
- [SEvenLLM: Benchmarking, Eliciting, and Enhancing Abilities of Large Language Models in Cyber Threat Intelligence](https://mlsanity.com/paper/2405.03446/) (2024)
- [CyberPal.AI: Empowering LLMs with Expert-Driven Cybersecurity Instructions](https://mlsanity.com/paper/2408.09304/) (2024)
- [Stackelberg vs. Nash in Security Games: An Extended Investigation of Interchangeability, Equivalence, and Uniqueness](https://mlsanity.com/paper/1401.3888/) (2014)
- [CyberRAG: An Agentic RAG cyber attack classification and reporting tool](https://mlsanity.com/paper/2507.02424/) (2025)
- [CyberAlly: Leveraging LLMs and Knowledge Graphs to Empower Cyber Defenders](https://mlsanity.com/paper/2504.07457/) (2025)
- [Comparing Decision Support Approaches for Cyber Security Investment](https://mlsanity.com/paper/1502.05532/) (2015)
- [Automated Post-Incident Policy Gap Analysis via Threat-Informed Evidence Mapping using Large Language Models](https://mlsanity.com/paper/2601.03287/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used in cybersecurity workflows, yet it remains unclear whether they can perform structured security reasoning or merely rely on superficial cues and prior knowledge. We study this question in the context of defence selection over attack graphs derived from real-world threat scenarios, including ransomware, supply-chain compromise, cloud abuse, Kubernetes attacks, POS malware, and ICS/OT intrusion. Given a budget constraint, LLMs must select security controls to minimise attacker success. We compare their strategies against each other and against a game-theoretic optimization baseline used as a normative reference for structured reasoning. Our results show that LLMs exhibit conditional competence. When explicit attack-graph structure is provided, they often produce coherent strategies close to the optimization baseline. However, their capabilities are fragile. LLM behaviour becomes increasingly fragile with graph complexity and is highly sensitive to framing. Small prompt changes can substantially alter rankings, and merely relabeling a poor strategy as ``optimal'' dramatically improves its evaluation. We further observe a non-monotonic relationship between formal risk and LLM judgement: strategies closest to the optimum are not necessarily ranked highest by LLM evaluators. To further probe reasoning ability, we ask LLMs to generate solvers for the same optimization problem. While the generated implementations recover the correct high-level formulation, they scale poorly compared to a purpose-built solver. Overall, our findings show that LLMs can approximate structured cybersecurity reasoning under controlled representations, but do not apply it robustly. This has important implications for the design and evaluation of AI-assisted security decision-support systems.