mlsanityData to 14 Oct 2026

Tools and Benchmarks for Automated Log Parsing

arXiv 1811.03509 ↗First posted 8 Nov 2018cs.SEcited by 56

Not on the map. Why?

Abstract (arXiv)

Logs are imperative in the development and maintenance process of many software systems. They record detailed runtime information that allows developers and support engineers to monitor their systems and dissect anomalous behaviors and errors. The increasing scale and complexity of modern software systems, however, make the volume of logs explodes. In many cases, the traditional way of manual log inspection becomes impractical. Many recent studies, as well as industrial tools, resort to powerful text search and machine learning-based analytics solutions. Due to the unstructured nature of logs, a first crucial step is to parse log messages into structured data for subsequent analysis. In recent years, automated log parsing has been widely studied in both academia and industry, producing a series of log parsers by different techniques. To better understand the characteristics of these log parsers, in this paper, we present a comprehensive evaluation study on automated log parsing and further release the tools and benchmarks for easy reuse. More specifically, we evaluate 13 log parsers on a total of 16 log datasets spanning distributed systems, supercomputers, operating systems, mobile systems, server applications, and standalone software. We report the benchmarking results in terms of accuracy, robustness, and efficiency, which are of practical importance when deploying automated log parsing in production. We also share the success stories and lessons learned in an industrial application at Huawei. We believe that our work could serve as the basis and provide valuable guidance to future research and deployment of automated log parsing.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 9; 2022: 7; 2023: 12; 2024: 16; 2025: 4; 2026: 8.

56 papers.

Sort:

By area:

  1. 2021Log-based Anomaly Detection Without Log Parsingcited by 59
  2. 2022Log-based Anomaly Detection with Deep Learning: How Far Are We?cited by 57
  3. 2021Experience Report: Deep Learning-based System Log Analysis for Anomaly Detectioncited by 37
  4. 2024Automatic Programming: Large Language Models and Beyondcited by 36
  5. 2022Deep Learning for Anomaly Detection in Log Data: A Surveycited by 28
  6. 2024LogLLM: Log-based Anomaly Detection Using Large Language Modelscited by 22
  7. 2024Building AI Agents for Autonomous Clouds: Challenges and Design Principlescited by 19
  8. 2023Log Parsing: How Far Can ChatGPT Go?cited by 16
  9. 2024OptLLM: Optimal Assignment of Queries to Large Language Modelscited by 16
  10. 2024LogLM: From Task-based to Instruction-based Automated Log Analysiscited by 15
  11. 2021Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Modelscited by 12
  12. 2023An Assessment of ChatGPT on Log Datacited by 12
38 more
  1. 2021UniLog: Deploy One Model and Specialize it for All Log Analysis Taskscited by 11
  2. 2024LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysiscited by 11
  3. 2023Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attentioncited by 9
  4. 2025Large Language Models for Security Operations Centers: A Comprehensive Surveycited by 9
  5. 2022LogGD:Detecting Anomalies from System Logs by Graph Neural Networkscited by 8
  6. 2023Graph Neural Networks based Log Anomaly Detection and Explanationcited by 8
  7. 2024Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledgecited by 8
  8. 2025Log Anomaly Detection with Large Language Models via Knowledge-Enriched Fusioncited by 8
  9. 2021OneLog: Towards End-to-End Training in Software Log Anomaly Detectioncited by 7
  10. 2021A Comprehensive Survey of Logging in Software: From Logging Statements Automation to Log Mining and Analysiscited by 6
  11. 2023On the Effectiveness of Log Representation for Log-based Anomaly Detectioncited by 6
  12. 2023Learning Representations on Logs for AIOpscited by 6
  13. 2024Demystifying and Extracting Fault-indicating Information from Logs for Failure Diagnosiscited by 6
  14. 2024A Comparative Study on Large Language Models for Log Parsingcited by 5
  15. 2021Multi-Source Anomaly Detection in Distributed IT Systemscited by 4
  16. 2024LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logscited by 4
  17. 2024Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correctioncited by 4
  18. 2022Feature Selection for Fault Detection and Prediction based on Event Log Analysiscited by 3
  19. 2023Log-based Anomaly Detection based on EVT Theory with feedbackcited by 3
  20. 2024LibreLog: Accurate and Efficient Unsupervised Log Parsing Using Open-Source Large Language Modelscited by 3
  21. 2025System Log Parsing with Large Language Models: A Reviewcited by 3
  22. 2023MoniLog: An Automated Log-Based Anomaly Detection System for Cloud Computing Infrastructurescited by 2
  23. 2024LUK: Empowering Log Understanding with Expert Knowledge from Large Language Modelscited by 2
  24. 2026Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safetycited by 2
  25. 2021Linnaeus: A highly reusable and adaptable ML based log classification pipelinecited by 1
  26. 2021FlexParser -- the adaptive log file parser for continuous results in a changing worldcited by 1
  27. 2022Leveraging Log Instructions in Log-based Anomaly Detectioncited by 1
  28. 2023Try with Simpler -- An Evaluation of Improved Principal Component Analysis in Log-based Anomaly Detectioncited by 1
  29. 2023LogLead -- Fast and Integrated Log Loader, Enhancer, and Anomaly Detectorcited by 1
  30. 2025A Novel GPT-Based Framework for Anomaly Detection in System Logscited by 1
  31. 2026LLM-Enhanced Log Anomaly Detection: A Comprehensive Benchmark of Large Language Models for Automated System Diagnosticscited by 1
  32. 2026TriCalRAG: A Three-Strategy, Retrieval-Augmented Benchmark for On-Premise LLM-Based Root Cause Analysis in AIOpscited by 1
  33. 2022Failure Identification from Unstable Log Data using Deep Learningcited by 0
  34. 2022LogAnMeta: Log Anomaly Detection Using Meta Learningcited by 0
  35. 2023ClusterLog: Clustering Logs for Effective Log-based Anomaly Detectioncited by 0
  36. 2023Interactive Log Parsing via Light-weight User Feedbackcited by 0
  37. 2024MLAD: A Unified Model for Multi-system Log Anomaly Detectioncited by 0
  38. 2024What Information Contributes to Log-based Anomaly Detection? Insights from a Configurable Transformer-Based Approachcited by 0

50 most cited of 56 All 56 (JSON)