# System Log Parsing with Large Language Models: A Review

arXiv: https://arxiv.org/abs/2504.04877 · HTML: https://mlsanity.com/paper/2504.04877/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 80 references that are in the map.
- First posted on arXiv 7 Apr 2025.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks](https://mlsanity.com/paper/2502.08235/) (2025)
- [Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluation](https://mlsanity.com/paper/2305.16938/) (2023)
- [Rethinking LLM Memorization through the Lens of Adversarial Compression](https://mlsanity.com/paper/2404.15146/) (2024)
- [Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics](https://mlsanity.com/paper/2008.06448/) (2020)
- [Tools and Benchmarks for Automated Log Parsing](https://mlsanity.com/paper/1811.03509/) (2018)
- [A Survey on Automated Log Analysis for Reliability Engineering](https://mlsanity.com/paper/2009.07237/) (2020)
- [Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies](https://mlsanity.com/paper/2308.07610/) (2023)
- [Scaling Laws for Fact Memorization of Large Language Models](https://mlsanity.com/paper/2406.15720/) (2024)
- [LILAC: Log Parsing using LLMs with Adaptive Parsing Cache](https://mlsanity.com/paper/2310.01796/) (2023)
- [LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing](https://mlsanity.com/paper/2404.18001/) (2024)
- [CodeFusion: A Pre-trained Diffusion Model for Code Generation](https://mlsanity.com/paper/2310.17680/) (2023)
- [Deep Learning for Anomaly Detection in Log Data: A Survey](https://mlsanity.com/paper/2207.03820/) (2022)
- [UniParser: A Unified Log Parser for Heterogeneous Log Data](https://mlsanity.com/paper/2202.06569/) (2022)
- [A Large-Scale Evaluation for Log Parsing Techniques: How Far Are We?](https://mlsanity.com/paper/2308.10828/) (2023)
- [Self-Supervised Log Parsing](https://mlsanity.com/paper/2003.07905/) (2020)
- [A large-scale study on research code quality and execution](https://mlsanity.com/paper/2103.12793/) (2021)
- [Did We Miss Something Important? Studying and Exploring Variable-Aware Log Abstraction](https://mlsanity.com/paper/2304.11391/) (2023)
- [Log Parsing: How Far Can ChatGPT Go?](https://mlsanity.com/paper/2306.01590/) (2023)
- [Lemur: Log Parsing with Entropy Sampling and Chain-of-Thought Merging](https://mlsanity.com/paper/2402.18205/) (2024)
- [LLM-based event log analysis techniques: A survey](https://mlsanity.com/paper/2502.00677/) (2025)
- [LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysis](https://mlsanity.com/paper/2407.01896/) (2024)
- [Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge](https://mlsanity.com/paper/2412.01377/) (2024)
- [LogShrink: Effective Log Compression by Leveraging Commonality and Variability of Log Data](https://mlsanity.com/paper/2309.09479/) (2023)
- [Log Parsing Evaluation in the Era of Modern Software Systems](https://mlsanity.com/paper/2308.09003/) (2023)
- [ECLIPSE: Semantic Entropy-LCS for Cross-Lingual Industrial Log Parsing](https://mlsanity.com/paper/2405.13548/) (2024)

## Built on by (most cited first, 3 of 3)

- [CAM-LDS: Cyber Attack Manifestations for Automatic Interpretation of System Logs and Security Alerts](https://mlsanity.com/paper/2603.04186/) (2026)
- [SCOPE: Tree-based Self-Correcting Online Log Parsing via Syntactic-Semantic Collaboration](https://mlsanity.com/paper/2603.27247/) (2026)
- [Just Testing, Move Along: Evasion of LLM-based System Log Interpretation by Prompt Injection](https://mlsanity.com/paper/2607.24174/) (2026)

## Abstract (arXiv)

Log data provides crucial insights for tasks like monitoring, root cause analysis, and anomaly detection. Due to the vast volume of logs, automated log parsing is essential to transform semi-structured log messages into structured representations. Recent advances in large language models (LLMs) have introduced the new research field of LLM-based log parsing. Despite promising results, there is no structured overview of the approaches in this relatively new research field with the earliest advances published in late 2023. This work systematically reviews 29 LLM-based log parsing methods. We benchmark seven of them on public datasets and critically assess their comparability and the reproducibility of their reported results. Our findings summarize the advances of this new research field, with insights on how to report results, which data sets, metrics and which terminology to use, and which inconsistencies to avoid, with code and results made publicly available for transparency.