A Survey on Automated Log Analysis for Reliability Engineering
Not on the map. Why?
Abstract (arXiv)
Logs are semi-structured text generated by logging statements in software source code. In recent decades, software logs have become imperative in the reliability assurance mechanism of many software systems because they are often the only data available that record software runtime information. As modern software is evolving into a large scale, the volume of logs has increased rapidly. To enable effective and efficient usage of modern software logs in reliability engineering, a number of studies have been conducted on automated log analysis. This survey presents a detailed overview of automated log analysis research, including how to automate and assist the writing of logging statements, how to compress logs, how to parse logs into structured event templates, and how to employ logs to detect anomalies, predict failures, and facilitate diagnosis. Additionally, we survey work that releases open-source toolkits and datasets. Based on the discussion of the recent advances, we present several promising future directions toward real-world and next-generation automated log analysis.
Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Sep 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 4; 2022: 3; 2023: 10; 2024: 13; 2025: 13; 2026: 7.
50 papers.
Sort:
By area:
- 2022Log-based Anomaly Detection with Deep Learning: How Far Are We?cited by 56
- 2021Experience Report: Deep Learning-based System Log Analysis for Anomaly Detectioncited by 37
- 2023LogGPT: Exploring ChatGPT for Log-Based Anomaly Detectioncited by 33
- 2024LLMParser: An Exploratory Study on Using Large Language Models for Log Parsingcited by 32
- 2024Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logscited by 17
- 2024Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Mergingcited by 16
- 2023AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challengescited by 13
- 2021UniLog: Deploy One Model and Specialize it for All Log Analysis Taskscited by 11
- 2025Large Language Models for Security Operations Centers: A Comprehensive Surveycited by 9
- 2023Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attentioncited by 8
- 2021A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysiscited by 6
- 2023On the Effectiveness of Log Representation for Log-based Anomaly Detectioncited by 6
38 more
- 2023Learning Representations on Logs for AIOpscited by 6
- 2024Demystifying and Extracting Fault-indicating Information from Logs for Failure Diagnosiscited by 6
- 2021A Taxonomy of Anomalies in Log Datacited by 5
- 2024ECLIPSE: Semantic Entropy-LCS for Cross-Lingual Industrial Log Parsingcited by 5
- 2024LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logscited by 4
- 2025AdaptiveLog: An Adaptive Log Analysis Framework with the Collaboration of Large and Small Language Modelcited by 4
- 2025Mapping the Landscape of Generative AI in Network Monitoring and Managementcited by 4
- 2024Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correctioncited by 3
- 2023LogAI: A Library for Log Analytics and Intelligencecited by 2
- 2024HELP: Hierarchical Embeddings-based Log Parsingcited by 2
- 2024LUK: Empowering Log Understanding with Expert Knowledge from Large Language Modelscited by 2
- 2025System Log Parsing with Large Language Models: A Reviewcited by 2
- 2025Deep Learning-based Intrusion Detection Systems: A Surveycited by 2
- 2025SchemaCoder: Automatic Log Schema Extraction Coder with Residual Q-Tree Boostingcited by 2
- 2025Hypergraph-Guided Regex Filter Synthesis for Event-Based Anomaly Detectioncited by 2
- 2025AOI: Context-Aware Multi-Agent Operations via Dynamic Scheduling and Hierarchical Memory Compressioncited by 2
- 2022Leveraging Log Instructions in Log-based Anomaly Detectioncited by 1
- 2023Practical Anomaly Detection over Multivariate Monitoring Metrics for Online Servicescited by 1
- 2023Try with Simpler -- An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detectioncited by 1
- 2025ClusterRCA: An End-to-End Approach for Network Fault Localization and Classification for HPC Systemcited by 1
- 2025ALPHA: LLM-Enabled Active Learning for Human-Free Network Anomaly Detectioncited by 1
- 2025Task-Aware Reduction for Scalable LLM-Database Systemscited by 1
- 2026LogSieve: Task-Aware CI Log Reduction for Sustainable LLM-Based Analysiscited by 1
- 2022Failure Identification from Unstable Log Data using Deep Learningcited by 0
- 2023Interactive Log Parsing via Light-weight User Feedbackcited by 0
- 2023A Multi-Level, Multi-Scale Visual Analytics Approach to Assessment of Multifidelity HPC Systemscited by 0
- 2024Semi-supervised learning via DQN for log anomaly detectioncited by 0
- 2024MTAD: Tools and Benchmarks for Multivariate Time Series Anomaly Detectioncited by 0
- 2024What Information Contributes to Log-based Anomaly Detection? Insights from a Configurable Transformer-Based Approachcited by 0
- 2024TPLogAD: Unsupervised Log Anomaly Detection Based on Event Templates and Key Parameterscited by 0
- 2025AutoLogger: A Multi-Agent Framework for the End-to-End Automated Loggingcited by 0
- 2025MINES: Explainable Anomaly Detection through Web API Invariant Inferencecited by 0
- 2026MicLog: Towards Accurate and Efficient LLM-based Log Parsing via Progressive Meta In-Context Learningcited by 0
- 2026Testing Storage-System Correctness: Challenges, Fuzzing Limitations, and AI-Augmented Opportunitiescited by 0
- 2026SCOPE: Tree-based Self-Correcting Online Log Parsing via Syntactic-Semantic Collaborationcited by 0
- 2026LLM-Based Automated Diagnosis Of Integration Test Failures At Googlecited by 0
- 2026FAME: Failure-Aware Mixture-of-Experts for Message-Level Log Anomaly Detectioncited by 0
- 2026A Query Engine for the Agentscited by 0
- 2026A Query Engine for the Agents27 May 2026
- 2026FAME: Failure-Aware Mixture-of-Experts for Message-Level Log Anomaly Detection21 May 2026
- 2026LLM-Based Automated Diagnosis Of Integration Test Failures At Google13 Apr 2026
- 2026SCOPE: Tree-based Self-Correcting Online Log Parsing via Syntactic-Semantic Collaboration28 Mar 2026
- 2026Testing Storage-System Correctness: Challenges, Fuzzing Limitations, and AI-Augmented Opportunities2 Feb 2026
- 2026LogSieve: Task-Aware CI Log Reduction for Sustainable LLM-Based Analysis28 Jan 2026
- 2026MicLog: Towards Accurate and Efficient LLM-based Log Parsing via Progressive Meta In-Context Learning11 Jan 2026
- 2025AOI: Context-Aware Multi-Agent Operations via Dynamic Scheduling and Hierarchical Memory Compression15 Dec 2025
- 2025MINES: Explainable Anomaly Detection through Web API Invariant Inference7 Dec 2025
- 2025AutoLogger: A Multi-Agent Framework for the End-to-End Automated Logging23 Nov 2025
- 2025Task-Aware Reduction for Scalable LLM-Database Systems13 Oct 2025
- 2025Large Language Models for Security Operations Centers: A Comprehensive Survey13 Sep 2025
38 more
- 2025Hypergraph-Guided Regex Filter Synthesis for Event-Based Anomaly Detection8 Sep 2025
- 2025ALPHA: LLM-Enabled Active Learning for Human-Free Network Anomaly Detection7 Sep 2025
- 2025SchemaCoder: Automatic Log Schema Extraction Coder with Residual Q-Tree Boosting25 Aug 2025
- 2025ClusterRCA: An End-to-End Approach for Network Fault Localization and Classification for HPC System17 Jun 2025
- 2025Deep Learning-based Intrusion Detection Systems: A Survey10 Apr 2025
- 2025System Log Parsing with Large Language Models: A Review7 Apr 2025
- 2025Mapping the Landscape of Generative AI in Network Monitoring and Management12 Feb 2025
- 2025AdaptiveLog: An Adaptive Log Analysis Framework with the Collaboration of Large and Small Language Model19 Jan 2025
- 2024TPLogAD: Unsupervised Log Anomaly Detection Based on Event Templates and Key Parameters22 Nov 2024
- 2024What Information Contributes to Log-based Anomaly Detection? Insights from a Configurable Transformer-Based Approach30 Sep 2024
- 2024Demystifying and Extracting Fault-indicating Information from Logs for Failure Diagnosis20 Sep 2024
- 2024LUK: Empowering Log Understanding with Expert Knowledge from Large Language Models3 Sep 2024
- 2024HELP: Hierarchical Embeddings-based Log Parsing15 Aug 2024
- 2024Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction5 Jun 2024
- 2024ECLIPSE: Semantic Entropy-LCS for Cross-Lingual Industrial Log Parsing22 May 2024
- 2024LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing27 Apr 2024
- 2024Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logs31 Mar 2024
- 2024Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Merging28 Feb 2024
- 2024LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs16 Feb 2024
- 2024MTAD: Tools and Benchmarks for Multivariate Time Series Anomaly Detection10 Jan 2024
- 2024Semi-supervised learning via DQN for log anomaly detection6 Jan 2024
- 2023LogGPT: Exploring ChatGPT for Log-Based Anomaly Detection3 Sep 2023
- 2023Try with Simpler -- An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detection24 Aug 2023
- 2023Practical Anomaly Detection over Multivariate Monitoring Metrics for Online Services19 Aug 2023
- 2023Learning Representations on Logs for AIOps18 Aug 2023
- 2023On the Effectiveness of Log Representation for Log-based Anomaly Detection17 Aug 2023
- 2023A Multi-Level, Multi-Scale Visual Analytics Approach to Assessment of Multifidelity HPC Systems15 Jun 2023
- 2023AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges10 Apr 2023
- 2023Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attention14 Feb 2023
- 2023LogAI: A Library for Log Analytics and Intelligence31 Jan 2023
- 2023Interactive Log Parsing via Light-weight User Feedback28 Jan 2023
- 2022Leveraging Log Instructions in Log-based Anomaly Detection7 Jul 2022
- 2022Failure Identification from Unstable Log Data using Deep Learning6 Apr 2022
- 2022Log-based Anomaly Detection with Deep Learning: How Far Are We?9 Feb 2022
- 2021UniLog: Deploy One Model and Specialize it for All Log Analysis Tasks6 Dec 2021
- 2021A Taxonomy of Anomalies in Log Data26 Nov 2021
- 2021A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysis24 Oct 2021
- 2021Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection13 Jul 2021
- 2024Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Mergingcited by 16
- 2026A Query Engine for the Agentscited by 0
- 2026A Query Engine for the Agents27 May 2026
- 2024Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Merging28 Feb 2024
- 2022Log-based Anomaly Detection with Deep Learning: How Far Are We?cited by 56
- 2021Experience Report: Deep Learning-based System Log Analysis for Anomaly Detectioncited by 37
- 2023LogGPT: Exploring ChatGPT for Log-Based Anomaly Detectioncited by 33
- 2024LLMParser: An Exploratory Study on Using Large Language Models for Log Parsingcited by 32
- 2024Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logscited by 17
- 2023AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challengescited by 13
- 2021UniLog: Deploy One Model and Specialize it for All Log Analysis Taskscited by 11
- 2025Large Language Models for Security Operations Centers: A Comprehensive Surveycited by 9
- 2023Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attentioncited by 8
- 2021A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysiscited by 6
- 2023On the Effectiveness of Log Representation for Log-based Anomaly Detectioncited by 6
- 2023Learning Representations on Logs for AIOpscited by 6
12 most cited of 44 All 44 (JSON)
- 2026LLM-Based Automated Diagnosis Of Integration Test Failures At Google13 Apr 2026
- 2026SCOPE: Tree-based Self-Correcting Online Log Parsing via Syntactic-Semantic Collaboration28 Mar 2026
- 2026Testing Storage-System Correctness: Challenges, Fuzzing Limitations, and AI-Augmented Opportunities2 Feb 2026
- 2026LogSieve: Task-Aware CI Log Reduction for Sustainable LLM-Based Analysis28 Jan 2026
- 2026MicLog: Towards Accurate and Efficient LLM-based Log Parsing via Progressive Meta In-Context Learning11 Jan 2026
- 2025MINES: Explainable Anomaly Detection through Web API Invariant Inference7 Dec 2025
- 2025Task-Aware Reduction for Scalable LLM-Database Systems13 Oct 2025
- 2025Large Language Models for Security Operations Centers: A Comprehensive Survey13 Sep 2025
- 2025Hypergraph-Guided Regex Filter Synthesis for Event-Based Anomaly Detection8 Sep 2025
- 2025ALPHA: LLM-Enabled Active Learning for Human-Free Network Anomaly Detection7 Sep 2025
- 2025SchemaCoder: Automatic Log Schema Extraction Coder with Residual Q-Tree Boosting25 Aug 2025
- 2025ClusterRCA: An End-to-End Approach for Network Fault Localization and Classification for HPC System17 Jun 2025
12 newest of 44 All 44 (JSON)