# Large Language Models Are Unreliable for Cyber Threat Intelligence

arXiv: https://arxiv.org/abs/2503.23175 · HTML: https://mlsanity.com/paper/2503.23175/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 72 references that are in the map.
- First posted on arXiv 29 Mar 2025.

## Builds on (most cited first, 30 of 36)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails](https://mlsanity.com/paper/2310.10501/) (2023)
- [Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models](https://mlsanity.com/paper/2305.19187/) (2023)
- [ReFT: Reasoning with Reinforced Fine-Tuning](https://mlsanity.com/paper/2401.08967/) (2024)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks](https://mlsanity.com/paper/2310.10844/) (2023)
- [Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation](https://mlsanity.com/paper/2305.16938/) (2023)
- [LM-Polygraph: Uncertainty Estimation for Language Models](https://mlsanity.com/paper/2311.07383/) (2023)
- [Benchmarking LLMs via Uncertainty Quantification](https://mlsanity.com/paper/2401.12794/) (2024)
- [The Good, The Bad, and The Greedy: Evaluation of LLMs Should Not Ignore Non-Determinism](https://mlsanity.com/paper/2407.10457/) (2024)
- [CyberMetric: A Benchmark Dataset based on Retrieval-Augmented Generation for Evaluating LLMs in Cybersecurity Knowledge](https://mlsanity.com/paper/2402.07688/) (2024)
- [Calibration and Correctness of Language Models for Code](https://mlsanity.com/paper/2402.02047/) (2024)
- [Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models](https://mlsanity.com/paper/2403.04786/) (2024)
- [Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs](https://mlsanity.com/paper/2412.13337/) (2024)
- [The Janus Interface: How Fine-Tuning in Large Language Models Amplifies the Privacy Risks](https://mlsanity.com/paper/2310.15469/) (2023)
- [Looking Beyond IoCs: Automatically Extracting Attack Patterns from External CTI](https://mlsanity.com/paper/2211.01753/) (2022)
- [SEvenLLM: Benchmarking, Eliciting, and Enhancing Abilities of Large Language Models in Cyber Threat Intelligence](https://mlsanity.com/paper/2405.03446/) (2024)
- [Assessing LLMs in Malicious Code Deobfuscation of Real-world Malware Campaigns](https://mlsanity.com/paper/2404.19715/) (2024)
- [Can We Trust LLMs? Mitigate Overconfidence Bias in LLMs through Knowledge Transfer](https://mlsanity.com/paper/2405.16856/) (2024)
- [AGIR: Automating Cyber Threat Intelligence Reporting with Natural Language Generation](https://mlsanity.com/paper/2310.02655/) (2023)
- [Actionable Cyber Threat Intelligence using Knowledge Graphs and Large Language Models](https://mlsanity.com/paper/2407.02528/) (2024)

## Built on by (most cited first, 5 of 5)

- [On LLMs' Internal Representation of Code Correctness](https://mlsanity.com/paper/2512.07404/) (2025)
- [Evaluating Open-Source LLMs for Multi-Label ATT&CK Technique Classification on CTI Reports](https://mlsanity.com/paper/2606.18166/) (2026)
- [RefinementEngine: Automating Intent-to-Device Filtering Policy Deployment under Network Constraints](https://mlsanity.com/paper/2604.01627/) (2026)
- [From Chasing Ghosts to Missed Attacks: Perspectives and Perceptions of SOC Practitioners on LLM Integration, Risks, and Readiness](https://mlsanity.com/paper/2608.00672/) (2026)
- [BEACON: Behavior-Anchored Cross-Source Knowledge Graph Construction for Cyber Threat Intelligence](https://mlsanity.com/paper/2608.28394/) (2026)

## Abstract (arXiv)

Several recent works have argued that Large Language Models (LLMs) can be used to tame the data deluge in the cybersecurity field, by improving the automation of Cyber Threat Intelligence (CTI) tasks. This work presents an evaluation methodology that other than allowing to test LLMs on CTI tasks when using zero-shot learning, few-shot learning and fine-tuning, also allows to quantify their consistency and their confidence level. We run experiments with three state-of-the-art LLMs and a dataset of 350 threat intelligence reports and present new evidence of potential security risks in relying on LLMs for CTI. We show how LLMs cannot guarantee sufficient performance on real-size reports while also being inconsistent and overconfident. Few-shot learning and fine-tuning only partially improve the results, thus posing doubts about the possibility of using LLMs for CTI scenarios, where labelled datasets are lacking and where confidence is a fundamental factor.