# The Use of Large Language Models (LLM) for Cyber Threat Intelligence (CTI) in Cybercrime Forums

arXiv: https://arxiv.org/abs/2408.03354 · HTML: https://mlsanity.com/paper/2408.03354/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 9 references that are in the map.
- First posted on arXiv 6 Aug 2024.

## Builds on (most cited first, 2 of 2)

- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [A Comprehensive Overview of Large Language Models (LLMs) for Cyber Defences: Opportunities and Directions](https://mlsanity.com/paper/2405.14487/) (2024)

## Built on by (most cited first, 6 of 6)

- [Uncovering Vulnerabilities of LLM-Assisted Cyber Threat Intelligence](https://mlsanity.com/paper/2509.23573/) (2025)
- [Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities](https://mlsanity.com/paper/2509.06921/) (2025)
- [CyberThreat-Eval: Can Large Language Models Automate Real-World Threat Research?](https://mlsanity.com/paper/2603.09452/) (2026)
- [DMFI: A Dual-Modality Log Analysis Framework for Insider Threat Detection with LoRA-Tuned Language Models](https://mlsanity.com/paper/2508.05694/) (2025)
- [An LLM-Powered AI Agent Framework for Holistic IoT Traffic Interpretation](https://mlsanity.com/paper/2510.13925/) (2025)
- [Assessing Crime Disclosure Patterns in a Large-Scale Cybercrime Forum](https://mlsanity.com/paper/2603.01624/) (2026)

## Abstract (arXiv)

Large language models (LLMs) can be used to analyze cyber threat intelligence (CTI) data from cybercrime forums, which contain extensive information and key discussions about emerging cyber threats. However, to date, the level of accuracy and efficiency of LLMs for such critical tasks has yet to be thoroughly evaluated. Hence, this study assesses the performance of an LLM system built on the OpenAI GPT-3.5-turbo model [8] to extract CTI information. To do so, a random sample of more than 700 daily conversations from three cybercrime forums - XSS, Exploit_in, and RAMP - was extracted, and the LLM system was instructed to summarize the conversations and predict 10 key CTI variables, such as whether a large organization and/or a critical infrastructure is being targeted, with only simple human-language instructions. Then, two coders reviewed each conversation and evaluated whether the information extracted by the LLM was accurate. The LLM system performed well, with an average accuracy score of 96.23%, an average precision of 90% and an average recall of 88.2%. Various ways to enhance the model were uncovered, such as the need to help the LLM distinguish between stories and past events, as well as being careful with verb tenses in prompts. Nevertheless, the results of this study highlight the relevance of using LLMs for cyber threat intelligence.