# LogBERT: Log Anomaly Detection via BERT

arXiv: https://arxiv.org/abs/2103.04475 · HTML: https://mlsanity.com/paper/2103.04475/

## Key facts

- Cited by 118 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2021.

## Built on by (most cited first, 30 of 118)

- [RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models](https://mlsanity.com/paper/2310.16340/) (2023)
- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [LAnoBERT: System Log Anomaly Detection based on BERT Masked Language Model](https://mlsanity.com/paper/2111.09564/) (2021)
- [Deep Learning for Anomaly Detection in Log Data: A Survey](https://mlsanity.com/paper/2207.03820/) (2022)
- [LogGPT: Log Anomaly Detection via GPT](https://mlsanity.com/paper/2309.14482/) (2023)
- [mABC: multi-Agent Blockchain-Inspired Collaboration for root cause analysis in micro-services architecture](https://mlsanity.com/paper/2404.12135/) (2024)
- [LogLLM: Log-based Anomaly Detection Using Large Language Models](https://mlsanity.com/paper/2411.08561/) (2024)
- [Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey](https://mlsanity.com/paper/2408.07583/) (2024)
- [Tele-Knowledge Pre-training for Fault Analysis](https://mlsanity.com/paper/2210.11298/) (2022)
- [CAN-BERT do it? Controller Area Network Intrusion Detection System based on BERT Language Model](https://mlsanity.com/paper/2210.09439/) (2022)
- [AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges](https://mlsanity.com/paper/2304.04661/) (2023)
- [MAD: Self-Supervised Masked Anomaly Detection Task for Multivariate Time Series](https://mlsanity.com/paper/2205.02100/) (2022)
- [UniLog: Deploy One Model and Specialize it for All Log Analysis Tasks](https://mlsanity.com/paper/2112.03159/) (2021)
- [TrajAudit: Automated Failure Diagnosis for Agentic Coding Systems](https://mlsanity.com/paper/2605.26563/) (2026)
- [A2Log: Attentive Augmented Log Anomaly Detection](https://mlsanity.com/paper/2109.09537/) (2021)
- [A Survey on XAI for 5G and Beyond Security: Technical Aspects, Challenges and Research Directions](https://mlsanity.com/paper/2204.12822/) (2022)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [LogLG: Weakly Supervised Log Anomaly Detection via Log-Event Graph Construction](https://mlsanity.com/paper/2208.10833/) (2022)
- [Graph Neural Networks based Log Anomaly Detection and Explanation](https://mlsanity.com/paper/2307.00527/) (2023)
- [A Critical Review of Common Log Data Sets Used for Evaluation of Sequence-based Anomaly Detection Techniques](https://mlsanity.com/paper/2309.02854/) (2023)
- [Reducing Events to Augment Log-based Anomaly Detection Models: An Empirical Study](https://mlsanity.com/paper/2409.04834/) (2024)
- [Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge](https://mlsanity.com/paper/2412.01377/) (2024)
- [OneLog: Towards End-to-End Training in Software Log Anomaly Detection](https://mlsanity.com/paper/2104.07324/) (2021)
- [LLMcap: Large Language Model for Unsupervised PCAP Failure Detection](https://mlsanity.com/paper/2407.06085/) (2024)
- [Learning Representations on Logs for AIOps](https://mlsanity.com/paper/2308.11526/) (2023)
- [Large Language Models for Anomaly Detection in Computational Workflows: from Supervised Fine-Tuning to In-Context Learning](https://mlsanity.com/paper/2407.17545/) (2024)
- [K4: Online Log Anomaly Detection Via Unsupervised Typicality Learning](https://mlsanity.com/paper/2507.20051/) (2025)
- [A Taxonomy of Anomalies in Log Data](https://mlsanity.com/paper/2111.13462/) (2021)
- [A Comparative Study on Large Language Models for Log Parsing](https://mlsanity.com/paper/2409.02474/) (2024)

## Abstract (arXiv)

Detecting anomalous events in online computer systems is crucial to protect the systems from malicious attacks or malfunctions. System logs, which record detailed information of computational events, are widely used for system status analysis. In this paper, we propose LogBERT, a self-supervised framework for log anomaly detection based on Bidirectional Encoder Representations from Transformers (BERT). LogBERT learns the patterns of normal log sequences by two novel self-supervised training tasks and is able to detect anomalies where the underlying patterns deviate from normal log sequences. The experimental results on three log datasets show that LogBERT outperforms state-of-the-art approaches for anomaly detection.