mlsanityData to 14 Oct 2026

System Log Parsing with Large Language Models: A Review

arXiv 2504.04877 ↗First posted 7 Apr 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Log data provides crucial insights for tasks like monitoring, root cause analysis, and anomaly detection. Due to the vast volume of logs, automated log parsing is essential to transform semi-structured log messages into structured representations. Recent advances in large language models (LLMs) have introduced the new research field of LLM-based log parsing. Despite promising results, there is no structured overview of the approaches in this relatively new research field with the earliest advances published in late 2023. This work systematically reviews 29 LLM-based log parsing methods. We benchmark seven of them on public datasets and critically assess their comparability and the reproducibility of their reported results. Our findings summarize the advances of this new research field, with insights on how to report results, which data sets, metrics and which terminology to use, and which inconsistencies to avoid, with code and results made publicly available for transparency.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 35 of its 80 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

35 of 80 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  4. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  5. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  6. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Taskscited by 152
  7. 2023Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluationcited by 136
  8. 2024Rethinking LLM Memorization through the Lens of Adversarial Compressioncited by 94
  9. 2020Loghub: A Large Collection of System Log Datasets for AI-driven Log Analyticscited by 70
  10. 2018Tools and Benchmarks for Automated Log Parsingcited by 56
  11. 2020A Survey on Automated Log Analysis for Reliability Engineeringcited by 50
  12. 2023Interpretable Online Log Analysis Using Large Language Models with Prompt Strategiescited by 38
23 more
  1. 2024Scaling Laws for Fact Memorization of Large Language Modelscited by 33
  2. 2023LILAC: Log Parsing using LLMs with Adaptive Parsing Cachecited by 32
  3. 2024LLMParser: An Exploratory Study on Using Large Language Models for Log Parsingcited by 32
  4. 2023CodeFusion: A Pre-trained Diffusion Model for Code Generationcited by 31
  5. 2022Deep Learning for Anomaly Detection in Log Data: A Surveycited by 28
  6. 2022UniParser: A Unified Log Parser for Heterogeneous Log Datacited by 26
  7. 2023A Large-Scale Evaluation for Log Parsing Techniques: How Far Are We?cited by 26
  8. 2020Self-Supervised Log Parsingcited by 23
  9. 2021A large-scale study on research code quality and executioncited by 22
  10. 2023Did We Miss Something Important? Studying and Exploring Variable-Aware Log Abstractioncited by 19
  11. 2023Log Parsing: How Far Can ChatGPT Go?cited by 16
  12. 2024Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Mergingcited by 16
  13. 2025LLM-based event log analysis techniques: A surveycited by 16
  14. 2024LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysiscited by 11
  15. 2024Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledgecited by 8
  16. 2023LogShrink: Effective Log Compression by Leveraging Commonality and Variability of Log Datacited by 7
  17. 2023Log Parsing Evaluation in the Era of Modern Software Systemscited by 6
  18. 2024ECLIPSE: Semantic Entropy-LCS for Cross-Lingual Industrial Log Parsingcited by 5
  19. 2024A Comparative Study on Large Language Models for Log Parsingcited by 5
  20. 2024Using Large Language Models for Template Detection from Security Event Logscited by 3
  21. 2024LUNAR: Unsupervised LLM-based Log Parsingcited by 2
  22. 2024HELP: Hierarchical Embeddings-based Log Parsingcited by 2
  23. 2024LogBabylon: A Unified Framework for Cross-Log File Integration and Analysiscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026CAM-LDS: Cyber Attack Manifestations for Automatic Interpretation of System Logs and Security Alertscited by 2
  2. 2026SCOPE: Tree-based Self-Correcting Online Log Parsing via Syntactic-Semantic Collaborationcited by 0
  3. 2026Just Testing, Move Along: Evasion of LLM-based System Log Interpretation by Prompt Injectioncited by 0