# LogLLM: Log-based Anomaly Detection Using Large Language Models

arXiv: https://arxiv.org/abs/2411.08561 · HTML: https://mlsanity.com/paper/2411.08561/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 69 references that are in the map.
- First posted on arXiv 13 Nov 2024.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [LogBERT: Log Anomaly Detection via BERT](https://mlsanity.com/paper/2103.04475/) (2021)
- [Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics](https://mlsanity.com/paper/2008.06448/) (2020)
- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [Tools and Benchmarks for Automated Log Parsing](https://mlsanity.com/paper/1811.03509/) (2018)
- [LogGPT: Exploring ChatGPT for Log-Based Anomaly Detection](https://mlsanity.com/paper/2309.01189/) (2023)
- [LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model](https://mlsanity.com/paper/2111.09564/) (2021)
- [Log Parsing with Prompt-based Few-shot Learning](https://mlsanity.com/paper/2302.07435/) (2023)
- [RAGLog: Log Anomaly Detection using Retrieval Augmented Generation](https://mlsanity.com/paper/2311.05261/) (2023)
- [LogGD:Detecting Anomalies from System Logs by Graph Neural Networks](https://mlsanity.com/paper/2209.07869/) (2022)
- [OneLog: Towards End-to-End Training in Software Log Anomaly Detection](https://mlsanity.com/paper/2104.07324/) (2021)
- [DABL: Detecting Semantic Anomalies in Business Processes Using Large Language Models](https://mlsanity.com/paper/2406.15781/) (2024)
- [FastLogAD: Log Anomaly Detection with Mask-Guided Pseudo Anomaly Generation and Discrimination](https://mlsanity.com/paper/2404.08750/) (2024)

## Built on by (most cited first, 22 of 22)

- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [Log Anomaly Detection with Large Language Models via Knowledge-Enriched Fusion](https://mlsanity.com/paper/2512.11997/) (2025)
- [K4: Online Log Anomaly Detection Via Unsupervised Typicality Learning](https://mlsanity.com/paper/2507.20051/) (2025)
- [FALCON: Transforming Cyber Threat Intelligence into Deployable IDS Rules with Self-Reflection](https://mlsanity.com/paper/2508.18684/) (2025)
- [Retrieval-Augmented LLMs for Security Incident Analysis](https://mlsanity.com/paper/2603.18196/) (2026)
- [Domain Specific Benchmarks for Evaluating Multimodal Large Language Models](https://mlsanity.com/paper/2506.12958/) (2025)
- [LogReasoner: Empowering LLMs with Expert-like Coarse-to-Fine Reasoning for Automated Log Analysis](https://mlsanity.com/paper/2509.20798/) (2025)
- [KORAL: Knowledge Graph Guided LLM Reasoning for SSD Operational Analysis](https://mlsanity.com/paper/2602.10246/) (2026)
- [ALPHA: LLM-Enabled Active Learning for Human-Free Network Anomaly Detection](https://mlsanity.com/paper/2509.05936/) (2025)
- [Towards Generalizable Context-aware Anomaly Detection: A Large-scale Benchmark in Cloud Environments](https://mlsanity.com/paper/2508.01844/) (2025)
- [Information-Dense Reasoning for Efficient and Auditable Security Alert Triage](https://mlsanity.com/paper/2512.08169/) (2025)
- [Automated Fault Detection in 5G Core Networks Using Large Language Models](https://mlsanity.com/paper/2512.19697/) (2025)
- [DP-FlogTinyLLM: Differentially private federated log anomaly detection using Tiny LLMs](https://mlsanity.com/paper/2604.19118/) (2026)
- [NLLog: Lightweight, Explainable SOC Anomaly Detection via Log-to-Language Rewriting](https://mlsanity.com/paper/2606.04957/) (2026)
- [Sample-Efficient LLM-Based Detection of Malicious Web Server Logs with Forensically Explainable Reasoning](https://mlsanity.com/paper/2606.08649/) (2026)
- [Can Large Language Models Reason about Event-Time Stream-Processing Semantics?](https://mlsanity.com/paper/2608.12348/) (2026)
- [Too Sure to Be Safe: Model Calibration for Reliable Log Anomaly Detection](https://mlsanity.com/paper/2608.17965/) (2026)
- [From Noise to Signal: Improving Security Log Anomaly Detection Using LLMs with Endpoint-Specific Logs](https://mlsanity.com/paper/2608.19938/) (2026)
- [Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systems](https://mlsanity.com/paper/2609.16599/) (2026)
- [TriFleetRCA: On-Premise LLM Root Cause Analysis for Kubernetes](https://mlsanity.com/paper/2609.23766/) (2026)
- [Towards Understanding LLM-Based Log Anomaly Detection: An Empirical Study of Performance, Efficiency, and Robustness](https://mlsanity.com/paper/2609.31371/) (2026)
- [Protocol-Sensitive Evaluation of Log Anomaly Detection: Component Costs and Target-Access Sensitivity on HDFS and BGL](https://mlsanity.com/paper/2610.05807/) (2026)

## Abstract (arXiv)

Software systems often record important runtime information in logs to help with troubleshooting. Log-based anomaly detection has become a key research area that aims to identify system issues through log data, ultimately enhancing the reliability of software systems. Existing methods often fall short in capturing the semantic information, typically expressed in natural language, or the sequential dependencies inherent in log sequences. In this paper, we propose LogLLM, a framework that enables collaboration between heterogeneous LLMs for log-based anomaly detection. LogLLM exploits the complementary capabilities of different LLM architectures: a Transformer encoder-based LLM is employed to extract fine-grained semantic vectors from individual log messages, while a Transformer decoder-based LLM is utilized to model sequential dependencies and generate anomaly detection decisions. To enable effective collaboration between these heterogeneous LLMs, we introduce a learnable projector to align their vector representation spaces. Furthermore, we design a progressive three-stage training strategy to optimize the collaboration between heterogeneous LLMs by gradually aligning their representations and adapting them to log anomaly detection. Unlike conventional methods that require log parsers to extract templates, LogLLM preprocesses log messages with regular expressions, streamlining the entire process. Experimental results on four public real-world datasets demonstrate that LogLLM outperforms state-of-the-art methods, achieving an average F1-score improvement of 6.6% over the strongest existing approach. Further analyses provide insights into the effectiveness of the key components of the model architecture and progressive three-stage training strategy.