# From Chasing Ghosts to Missed Attacks: Perspectives and Perceptions of SOC Practitioners on LLM Integration, Risks, and Readiness

arXiv: https://arxiv.org/abs/2608.00672 · HTML: https://mlsanity.com/paper/2608.00672/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 79 references that are in the map.
- First posted on arXiv 1 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Cognitive Bias in Decision-Making with LLMs](https://mlsanity.com/paper/2403.00811/) (2024)
- [Get my drift? Catching LLM Task Drift with Activation Deltas](https://mlsanity.com/paper/2406.00799/) (2024)
- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [Revolutionizing Cyber Threat Detection with Large Language Models: A privacy-preserving BERT-based Lightweight Model for IoT/IIoT Devices](https://mlsanity.com/paper/2306.14263/) (2023)
- [LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks](https://mlsanity.com/paper/2502.06215/) (2025)
- [A Comprehensive Overview of Large Language Models (LLMs) for Cyber Defences: Opportunities and Directions](https://mlsanity.com/paper/2405.14487/) (2024)
- [Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns](https://mlsanity.com/paper/2405.06371/) (2024)
- [Generative AI in Cybersecurity: A Comprehensive Review of LLM Applications and Vulnerabilities](https://mlsanity.com/paper/2405.12750/) (2024)
- [LLMs in the SOC: An Empirical Study of Human-AI Collaboration in Security Operations Centres](https://mlsanity.com/paper/2508.18947/) (2025)
- [Using LLMs to Automate Threat Intelligence Analysis Workflows in Security Operation Centers](https://mlsanity.com/paper/2407.13093/) (2024)
- [AI-Driven Guided Response for Security Operation Centers with Microsoft Copilot for Security](https://mlsanity.com/paper/2407.09017/) (2024)
- [An Assessment of the Usability of Machine Learning Based Tools for the Security Operations Center](https://mlsanity.com/paper/2012.09013/) (2020)
- [Large Language Models Are Unreliable for Cyber Threat Intelligence](https://mlsanity.com/paper/2503.23175/) (2025)
- [Towards Characterizing Cyber Networks with Large Language Models](https://mlsanity.com/paper/2411.07089/) (2024)

## Abstract (arXiv)

Security Operations Centers (SOCs) process large volumes of security events, requiring analysts to accurately detect and assess ongoing cyberattacks under time pressure. Recent advances in Large Language Models (LLMs) suggest potential benefits for security operations, yet their practical suitability for real-world SOC workflows remains poorly understood. To address this gap, we conducted 25 semi-structured interviews with SOC practitioners who had prior experience with LLMs, complemented by interactive scenarios to anticipate challenges and identify opportunities for the responsible integration of LLM-based tools into SOC workflows. We identified 15 LLM use cases grouped into six functional categories. While LLMs are valued for automating repetitive, low-level tasks such as report automation, practitioners rate high-impact tasks such as incident analysis as not yet feasible, reporting limitations in technical depth, context awareness, and organization-specific knowledge. They locate these limitations less in the models than in the readiness of their SOCs and human factors driving over-reliance. Despite concerns, practitioners express a strong willingness to adopt LLMs, describing competitive pressure that leaves few alternatives. This work contributes an empirical, practitioner-driven analysis of LLM use across SOC roles and organizations and derives concrete design and integration requirements for human-centered, operationally safe LLM-assisted security operations.