# Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection

arXiv: https://arxiv.org/abs/2107.05908 · HTML: https://mlsanity.com/paper/2107.05908/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 59 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [Tools and Benchmarks for Automated Log Parsing](https://mlsanity.com/paper/1811.03509/) (2018)
- [A Survey on Automated Log Analysis for Reliability Engineering](https://mlsanity.com/paper/2009.07237/) (2020)
- [Self-Attentive Classification-Based Anomaly Detection in Unstructured Logs](https://mlsanity.com/paper/2008.09340/) (2020)
- [Integrating Distributional Lexical Contrast into Word Embeddings for Antonym-Synonym Distinction](https://mlsanity.com/paper/1605.07766/) (2016)
- [Logram: Efficient Log Parsing Using n-Gram Dictionaries](https://mlsanity.com/paper/2001.03038/) (2020)
- [Log-based software monitoring: a systematic mapping study](https://mlsanity.com/paper/1912.05878/) (2019)

## Built on by (most cited first, 30 of 37)

- [Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code](https://mlsanity.com/paper/2311.07989/) (2023)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies](https://mlsanity.com/paper/2308.07610/) (2023)
- [Deep Learning for Anomaly Detection in Log Data: A Survey](https://mlsanity.com/paper/2207.03820/) (2022)
- [LogGPT: Log Anomaly Detection via GPT](https://mlsanity.com/paper/2309.14482/) (2023)
- [RAGLog: Log Anomaly Detection using Retrieval Augmented Generation](https://mlsanity.com/paper/2311.05261/) (2023)
- [AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges](https://mlsanity.com/paper/2304.04661/) (2023)
- [Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attention](https://mlsanity.com/paper/2302.06914/) (2023)
- [LogGD:Detecting Anomalies from System Logs by Graph Neural Networks](https://mlsanity.com/paper/2209.07869/) (2022)
- [Graph Neural Networks based Log Anomaly Detection and Explanation](https://mlsanity.com/paper/2307.00527/) (2023)
- [A Critical Review of Common Log Data Sets Used for Evaluation of Sequence-based Anomaly Detection Techniques](https://mlsanity.com/paper/2309.02854/) (2023)
- [On the Effectiveness of Log Representation for Log-based Anomaly Detection](https://mlsanity.com/paper/2308.08736/) (2023)
- [Demystifying and Extracting Fault-indicating Information from Logs for Failure Diagnosis](https://mlsanity.com/paper/2409.13561/) (2024)
- [LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs](https://mlsanity.com/paper/2402.10397/) (2024)
- [Behavioral Anomaly Detection in Distributed Systems via Federated Contrastive Learning](https://mlsanity.com/paper/2506.19246/) (2025)
- [Log-based Anomaly Detection based on EVT Theory with feedback](https://mlsanity.com/paper/2306.05032/) (2023)
- [LogLLaMA: Transformer-based log anomaly detection with LLaMA](https://mlsanity.com/paper/2503.14849/) (2025)
- [POLAR: Automating Cyber Threat Prioritization through LLM-Powered Assessment](https://mlsanity.com/paper/2510.01552/) (2025)
- [LLMLogAnalyzer: A Clustering-Based Log Analysis Chatbot using Large Language Models](https://mlsanity.com/paper/2510.24031/) (2025)
- [LogAI: A Library for Log Analytics and Intelligence](https://mlsanity.com/paper/2301.13415/) (2023)
- [Decodable and Sample Invariant Continuous Object Encoder](https://mlsanity.com/paper/2311.00187/) (2023)
- [Leveraging Log Instructions in Log-based Anomaly Detection](https://mlsanity.com/paper/2207.03206/) (2022)
- [NLP-based Cross-Layer 5G Vulnerabilities Detection via Fuzzing Generated Run-Time Profiling](https://mlsanity.com/paper/2305.08226/) (2023)
- [Try with Simpler -- An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detection](https://mlsanity.com/paper/2308.12612/) (2023)
- [Enhancing Reasoning Capacity of SLM using Cognitive Enhancement](https://mlsanity.com/paper/2404.01135/) (2024)
- [AnoGAN for Tabular Data: A Novel Approach to Anomaly Detection](https://mlsanity.com/paper/2405.03075/) (2024)
- [LogBabylon: A Unified Framework for Cross-Log File Integration and Analysis](https://mlsanity.com/paper/2412.12364/) (2024)
- [A Novel GPT-Based Framework for Anomaly Detection in System Logs](https://mlsanity.com/paper/2510.16044/) (2025)
- [Benchmarking Small Language Models and Small Reasoning Language Models on System Log Severity Classification](https://mlsanity.com/paper/2601.07790/) (2026)
- [AI based Log Analyser: A Practical Approach](https://mlsanity.com/paper/2203.10960/) (2022)

## Abstract (arXiv)

Logs have been an imperative resource to ensure the reliability and continuity of many software systems, especially large-scale distributed systems. They faithfully record runtime information to facilitate system troubleshooting and behavior understanding. Due to the large scale and complexity of modern software systems, the volume of logs has reached an unprecedented level. Consequently, for log-based anomaly detection, conventional manual inspection methods or even traditional machine learning-based methods become impractical, which serve as a catalyst for the rapid development of deep learning-based solutions. However, there is currently a lack of rigorous comparison among the representative log-based anomaly detectors that resort to neural networks. Moreover, the re-implementation process demands non-trivial efforts, and bias can be easily introduced. To better understand the characteristics of different anomaly detectors, in this paper, we provide a comprehensive review and evaluation of five popular neural networks used by six state-of-the-art methods. Particularly, four of the selected methods are unsupervised, and the remaining two are supervised. These methods are evaluated with two publicly available log datasets, which contain nearly 16 million log messages and 0.4 million anomaly instances in total. We believe our work can serve as a basis in this field and contribute to future academic research and industrial applications.