# Uncovering Vulnerabilities of LLM-Assisted Cyber Threat Intelligence

arXiv: https://arxiv.org/abs/2509.23573 · HTML: https://mlsanity.com/paper/2509.23573/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 131 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Builds on (most cited first, 30 of 37)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Training Language Models to Self-Correct via Reinforcement Learning](https://mlsanity.com/paper/2409.12917/) (2024)
- [Agent Laboratory: Using LLM Agents as Research Assistants](https://mlsanity.com/paper/2501.04227/) (2025)
- [Data Interpreter: An LLM Agent For Data Science](https://mlsanity.com/paper/2402.18679/) (2024)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [Jailbreaking Attack against Multimodal Large Language Model](https://mlsanity.com/paper/2402.02309/) (2024)
- [LLM Agents can Autonomously Exploit One-day Vulnerabilities](https://mlsanity.com/paper/2404.08144/) (2024)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation](https://mlsanity.com/paper/2402.14744/) (2024)
- [Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents](https://mlsanity.com/paper/2503.24047/) (2025)
- [Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification](https://mlsanity.com/paper/2407.20859/) (2024)
- [CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligence](https://mlsanity.com/paper/2406.07599/) (2024)
- [LLM Agents for Education: Advances and Applications](https://mlsanity.com/paper/2503.11733/) (2025)
- [Chainpoll: A high efficacy method for LLM hallucination detection](https://mlsanity.com/paper/2310.18344/) (2023)
- [Defending Pre-trained Language Models as Few-shot Learners against Backdoor Attacks](https://mlsanity.com/paper/2309.13256/) (2023)
- [Generalization-Enhanced Code Vulnerability Detection via Multi-Task Instruction Fine-Tuning](https://mlsanity.com/paper/2406.03718/) (2024)
- [Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Report](https://mlsanity.com/paper/2508.01059/) (2025)
- [Exploit Prediction Scoring System (EPSS)](https://mlsanity.com/paper/1908.04856/) (2019)
- [SEvenLLM: Benchmarking, Eliciting, and Enhancing Abilities of Large Language Models in Cyber Threat Intelligence](https://mlsanity.com/paper/2405.03446/) (2024)
- [BitsAI-CR: Automated Code Review via LLM in Practice](https://mlsanity.com/paper/2501.15134/) (2025)
- [Cyber-Zero: Training Cybersecurity Agents without Runtime](https://mlsanity.com/paper/2508.00910/) (2025)
- [An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliability](https://mlsanity.com/paper/2506.13639/) (2025)
- [Zodiac: A Cardiologist-Level LLM Framework for Multi-Agent Diagnostics](https://mlsanity.com/paper/2410.02026/) (2024)
- [Preemptive Detection and Correction of Misaligned Actions in LLM Agents](https://mlsanity.com/paper/2407.11843/) (2024)
- [Hackphyr: A Local Fine-Tuned LLM Agent for Network Security Environments](https://mlsanity.com/paper/2409.11276/) (2024)
- [Cyber Threat Intelligence Model: An Evaluation of Taxonomies, Sharing Standards, and Ontologies within Cyber Threat Intelligence](https://mlsanity.com/paper/2103.03530/) (2021)
- [LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures](https://mlsanity.com/paper/2505.01177/) (2025)
- [You Cannot Escape Me: Detecting Evasions of SIEM Rules in Enterprise Networks](https://mlsanity.com/paper/2311.10197/) (2023)
- [The Use of Large Language Models (LLM) for Cyber Threat Intelligence (CTI) in Cybercrime Forums](https://mlsanity.com/paper/2408.03354/) (2024)
- [Economic Factors of Vulnerability Trade and Exploitation](https://mlsanity.com/paper/1708.04866/) (2017)

## Built on by (most cited first, 6 of 6)

- [The Trap of Trajectory: Towards Understanding and Mitigating Spurious Correlations in Agentic Memory](https://mlsanity.com/paper/2605.09330/) (2026)
- [The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines](https://mlsanity.com/paper/2512.06659/) (2025)
- [A Proposal for an Agentic AI Architecture to Support Multi-Domain Decision-Making in the Brazilian Armed Forces](https://mlsanity.com/paper/2609.20080/) (2026)
- [Polar: LLM-Powered Synthesis of Real-World Cyber Evidence for Prioritization and Mitigation](https://mlsanity.com/paper/2610.07298/) (2026)
- [From Sandbox to Enforcement: Confidence-Qualified Threat Intelligence for Critical Infrastructure](https://mlsanity.com/paper/2610.07310/) (2026)
- [Understanding and Mitigating Inference-Time Overreliance Using Agentic Memory](https://mlsanity.com/paper/2610.07311/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used to help security analysts manage the surge of cyber threats, automating tasks from vulnerability assessment to incident response. Yet in operational CTI workflows, reliability gaps remain substantial. Existing explanations often point to generic model issues (e.g., hallucination), but we argue the dominant bottleneck is the threat landscape itself: CTI is heterogeneous, volatile, and fragmented. Under these conditions, evidence is intertwined, crowdsourced, and temporally unstable, which are properties that standard LLM-based studies rarely capture. In this paper, we present a comprehensive empirical study of LLM vulnerabilities in CTI reasoning. We introduce a human-in-the-loop categorization framework that robustly labels failure modes across the CTI lifecycle, avoiding the brittleness of automated "LLM-as-a-judge" pipelines. We identify three domain-specific cognitive failures: spurious correlations from superficial metadata, contradictory knowledge from conflicting sources, and constrained generalization to emerging threats. We validate these mechanisms via causal interventions and show that targeted defenses reduce failure rates significantly. Together, these results offer a concrete roadmap for building resilient, domain-aware CTI agents.