# Audit-LLM: Multi-Agent Collaboration for Log-based Insider Threat Detection

arXiv: https://arxiv.org/abs/2408.08902 · HTML: https://mlsanity.com/paper/2408.08902/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 48 references that are in the map.
- First posted on arXiv 12 Aug 2024.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 12 of 12)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [LLM Agents can Autonomously Exploit One-day Vulnerabilities](https://mlsanity.com/paper/2404.08144/) (2024)
- [AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks](https://mlsanity.com/paper/2403.01038/) (2024)
- [Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debate](https://mlsanity.com/paper/2402.07401/) (2024)
- [LogGPT: Exploring ChatGPT for Log-Based Anomaly Detection](https://mlsanity.com/paper/2309.01189/) (2023)
- [Benchmarking Large Language Models for Log Analysis, Security, and Interpretation](https://mlsanity.com/paper/2311.14519/) (2023)
- [Deep Learning for Insider Threat Detection: Review, Challenges and Opportunities](https://mlsanity.com/paper/2005.12433/) (2020)
- [Leveraging Partial Symmetry for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2401.00167/) (2023)
- [Crimson: Empowering Strategic Reasoning in Cybersecurity through Large Language Models](https://mlsanity.com/paper/2403.00878/) (2024)
- [LAN: Learning Adaptive Neighbors for Real-Time Insider Threat Detection](https://mlsanity.com/paper/2403.09209/) (2024)
- [Insider Threat Detection via Hierarchical Neural Temporal Point Processes](https://mlsanity.com/paper/1910.03171/) (2019)
- [PicoDomain: A Compact High-Fidelity Cybersecurity Dataset](https://mlsanity.com/paper/2008.09192/) (2020)

## Built on by (most cited first, 18 of 18)

- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety](https://mlsanity.com/paper/2502.05206/) (2025)
- [Security of Internet of Agents: Attacks and Countermeasures](https://mlsanity.com/paper/2505.08807/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration](https://mlsanity.com/paper/2603.04474/) (2026)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [Chimera: Harnessing Multi-Agent LLMs for Automatic Insider Threat Simulation](https://mlsanity.com/paper/2508.07745/) (2025)
- [Mapping Human Anti-collusion Mechanisms to Multi-agent AI Systems](https://mlsanity.com/paper/2601.00360/) (2026)
- [Retrieval-Augmented LLMs for Security Incident Analysis](https://mlsanity.com/paper/2603.18196/) (2026)
- [Foundation Models for Anomaly Detection: Vision and Challenges](https://mlsanity.com/paper/2502.06911/) (2025)
- [The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines](https://mlsanity.com/paper/2512.06659/) (2025)
- [DMFI: A Dual-Modality Log Analysis Framework for Insider Threat Detection with LoRA-Tuned Language Models](https://mlsanity.com/paper/2508.05694/) (2025)
- [Adversarial Attacks in Multi-Agent LLM Pipelines: Unveiling Structural Vulnerabilities in Agentic AI Architectures](https://mlsanity.com/paper/2608.00718/) (2026)
- [A Hybrid Insider Threat Detection Framework Combining Multi-Agent Simulation, Layered SIEM Correlation, and Theory-of-Mind Reasoning](https://mlsanity.com/paper/2601.04243/) (2026)
- [Next-generation cyberattack detection with large language models: anomaly analysis across heterogeneous logs](https://mlsanity.com/paper/2602.06777/) (2026)
- [Minos: A Multi-Agent Collaborative Framework for Provenance-Based Backward Tracking](https://mlsanity.com/paper/2607.00440/) (2026)
- [LLMs for Zero-Shot Threat Detection via Structured Risk Indicators](https://mlsanity.com/paper/2608.16508/) (2026)
- [From Investigation Failures to Reliable SOC Agents: Understanding and Improving LLM-Based Alert Triage](https://mlsanity.com/paper/2610.10608/) (2026)

## Abstract (arXiv)

Log-based insider threat detection (ITD) detects malicious user activities by auditing log entries. Recently, large language models (LLMs) with strong common sense knowledge have emerged in the domain of ITD. Nevertheless, diverse activity types and overlong log files pose a significant challenge for LLMs in directly discerning malicious ones within myriads of normal activities. Furthermore, the faithfulness hallucination issue from LLMs aggravates its application difficulty in ITD, as the generated conclusion may not align with user commands and activity context. In response to these challenges, we introduce Audit-LLM, a multi-agent log-based insider threat detection framework comprising three collaborative agents: (i) the Decomposer agent, breaking down the complex ITD task into manageable sub-tasks using Chain-of-Thought (COT) reasoning;(ii) the Tool Builder agent, creating reusable tools for sub-tasks to overcome context length limitations in LLMs; and (iii) the Executor agent, generating the final detection conclusion by invoking constructed tools. To enhance conclusion accuracy, we propose a pair-wise Evidence-based Multi-agent Debate (EMAD) mechanism, where two independent Executors iteratively refine their conclusions through reasoning exchange to reach a consensus. Comprehensive experiments conducted on three publicly available ITD datasets-CERT r4.2, CERT r5.2, and PicoDomain-demonstrate the superiority of our method over existing baselines and show that the proposed EMAD significantly improves the faithfulness of explanations generated by LLMs.