# Just Testing, Move Along: Evasion of LLM-based System Log Interpretation by Prompt Injection

arXiv: https://arxiv.org/abs/2607.24174 · HTML: https://mlsanity.com/paper/2607.24174/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security.
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 27 Jul 2026.

## Placed in

- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 14 of 14)

- [Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks](https://mlsanity.com/paper/2310.10844/) (2023)
- [Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies](https://mlsanity.com/paper/2308.07610/) (2023)
- [The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis](https://mlsanity.com/paper/2602.10453/) (2026)
- [LogLM: From Task-based to Instruction-based Automated Log Analysis](https://mlsanity.com/paper/2410.09352/) (2024)
- [LLMs in the SOC: An Empirical Study of Human-AI Collaboration in Security Operations Centres](https://mlsanity.com/paper/2508.18947/) (2025)
- [An Assessment of ChatGPT on Log Data](https://mlsanity.com/paper/2309.07938/) (2023)
- [Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge](https://mlsanity.com/paper/2412.01377/) (2024)
- [A Systematic Literature Review on LLM Defenses Against Prompt Injection and Jailbreaking: Expanding NIST Taxonomy](https://mlsanity.com/paper/2601.22240/) (2026)
- [SSCAE -- Semantic, Syntactic, and Context-aware natural language Adversarial Examples generator](https://mlsanity.com/paper/2403.11833/) (2024)
- [Evaluating Language Models For Threat Detection in IoT Security Logs](https://mlsanity.com/paper/2507.02390/) (2025)
- [OntoLogX: Ontology-Guided Knowledge Graph Extraction from Cybersecurity Logs with Large Language Models](https://mlsanity.com/paper/2510.01409/) (2025)
- [System Log Parsing with Large Language Models: A Review](https://mlsanity.com/paper/2504.04877/) (2025)
- [Adversarial Robustness for Machine Learning Cyber Defenses Using Log Data](https://mlsanity.com/paper/2007.14983/) (2020)
- [CAM-LDS: Cyber Attack Manifestations for Automatic Interpretation of System Logs and Security Alerts](https://mlsanity.com/paper/2603.04186/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly integrated into Security Operations Center (SOC) workflows, where they support analysts in tasks such as the interpretation of system logs. However, the ability of LLMs to directly process untrusted textual input also introduces new attack surfaces. In particular, attackers can inject contextual information or explicit instructions into log entries in order to influence how malicious activity is interpreted by the model. Despite the growing adoption of LLMs for log analytics, the robustness of such systems against adversarial log injection remains largely unexplored. To address this gap, this paper presents a framework for evaluating prompt injection attacks against LLM-based log interpretation. Using log traces generated during real cyber attacks, our approach creates adversarial examples through generic injection generation, refinement, and attack-specific optimization. Our evaluation across multiple state-of-the-art LLMs shows that these injections can cause malicious log traces to be classified as benign despite containing clear indicators of compromise. As a potential remedy, we show that the explanations generated by the LLMs alongside their classifications frequently contain indicators of adversarial manipulation that can be leveraged to detect such attacks.