# False Alarms, Real Damage: Adversarial Attacks Using LLM-based Models on Text-based Cyber Threat Intelligence Systems

arXiv: https://arxiv.org/abs/2507.06252 · HTML: https://mlsanity.com/paper/2507.06252/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 78 references that are in the map.
- First posted on arXiv 5 Jul 2025.

## Builds on (most cited first, 30 of 30)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Statistical Aspects of Wasserstein Distances](https://mlsanity.com/paper/1806.05500/) (2018)
- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods](https://mlsanity.com/paper/2210.07321/) (2022)
- [CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligence](https://mlsanity.com/paper/2406.07599/) (2024)
- [SecureBERT: A Domain-Specific Language Model for Cybersecurity](https://mlsanity.com/paper/2204.02685/) (2022)
- [A Survey of Black-Box Adversarial Attacks on Computer Vision Models](https://mlsanity.com/paper/1912.01667/) (2019)
- [Adversarial Deep Ensemble: Evasion Attacks and Defenses for Malware Detection](https://mlsanity.com/paper/2006.16545/) (2020)
- [Generating Fake Cyber Threat Intelligence Using Transformer-Based Models](https://mlsanity.com/paper/2102.04351/) (2021)
- [Looking Beyond IoCs: Automatically Extracting Attack Patterns from External CTI](https://mlsanity.com/paper/2211.01753/) (2022)
- [Machine Learning Security against Data Poisoning: Are We There Yet?](https://mlsanity.com/paper/2204.05986/) (2022)
- [EXTRACTOR: Extracting Attack Behavior from Threat Reports](https://mlsanity.com/paper/2104.08618/) (2021)
- [Generating Natural Language Adversarial Examples on a Large Scale with Generative Models](https://mlsanity.com/paper/2003.10388/) (2020)
- [Do Perceptually Aligned Gradients Imply Adversarial Robustness?](https://mlsanity.com/paper/2207.11378/) (2022)
- [What are the attackers doing now? Automating cyber threat intelligence extraction from text on pace with the changing threat landscape: A survey](https://mlsanity.com/paper/2109.06808/) (2021)
- [Evaluation of LLM Chatbots for OSINT-based Cyber Threat Awareness](https://mlsanity.com/paper/2401.15127/) (2024)
- [CyberPal.AI: Empowering LLMs with Expert-Driven Cybersecurity Instructions](https://mlsanity.com/paper/2408.09304/) (2024)
- [Processing Tweets for Cybersecurity Threat Awareness](https://mlsanity.com/paper/1904.02072/) (2019)
- [KGV: Integrating Large Language Models with Knowledge Graphs for Cyber Threat Intelligence Credibility Assessment](https://mlsanity.com/paper/2408.08088/) (2024)
- [Mining Threat Intelligence about Open-Source Projects and Libraries from Code Repository Issues and Bug Reports](https://mlsanity.com/paper/1808.04673/) (2018)
- [Cyberthreat Detection from Twitter using Deep Neural Networks](https://mlsanity.com/paper/1904.01127/) (2019)
- [A Novel Approach for Detection and Ranking of Trendy and Emerging Cyber Threat Events in Twitter Streams](https://mlsanity.com/paper/1907.07768/) (2019)
- [Towards Effective Identification of Attack Techniques in Cyber Threat Intelligence Reports using Large Language Models](https://mlsanity.com/paper/2505.03147/) (2025)
- [From Threat Reports to Continuous Threat Intelligence: A Comparison of Attack Technique Extraction Methods from Textual Artifacts](https://mlsanity.com/paper/2210.02601/) (2022)
- [Collecting Indicators of Compromise from Unstructured Text of Cybersecurity Articles using Neural-Based Sequence Labelling](https://mlsanity.com/paper/1907.02636/) (2019)
- [AnnoCTR: A Dataset for Detecting and Linking Entities, Tactics, and Techniques in Cyber Threat Reports](https://mlsanity.com/paper/2404.07765/) (2024)
- [EaTVul: ChatGPT-based Evasion Attack Against Software Vulnerability Detection](https://mlsanity.com/paper/2407.19216/) (2024)

## Built on by (most cited first, 2 of 2)

- [RAGRank: Using PageRank to Counter Poisoning in CTI LLM Pipelines](https://mlsanity.com/paper/2510.20768/) (2025)
- [Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability](https://mlsanity.com/paper/2607.10402/) (2026)

## Abstract (arXiv)

Cyber Threat Intelligence (CTI) has emerged as a vital complementary approach that operates in the early phases of the cyber threat lifecycle. CTI involves collecting, processing, and analyzing threat data to provide a more accurate and rapid understanding of cyber threats. Due to the large volume of data, automation through Machine Learning (ML) and Natural Language Processing (NLP) models is essential for effective CTI extraction. These automated systems leverage Open Source Intelligence (OSINT) from sources like social networks, forums, and blogs to identify Indicators of Compromise (IoCs). Although prior research has focused on adversarial attacks on specific ML models, this study expands the scope by investigating vulnerabilities within various components of the entire CTI pipeline and their susceptibility to adversarial attacks. These vulnerabilities arise because they ingest textual inputs from various open sources, including real and potentially fake content. We analyse three types of attacks against CTI pipelines, including evasion, flooding, and poisoning, and assess their impact on the system's information selection capabilities. Specifically, on fake text generation, the work demonstrates how adversarial text generation techniques can create fake cybersecurity and cybersecurity-like text that misleads classifiers, degrades performance, and disrupts system functionality. The focus is primarily on the evasion attack, as it precedes and enables flooding and poisoning attacks within the CTI pipeline.