# Log Parsing: How Far Can ChatGPT Go?

arXiv: https://arxiv.org/abs/2306.01590 · HTML: https://mlsanity.com/paper/2306.01590/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 15 of 15)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [Tools and Benchmarks for Automated Log Parsing](https://mlsanity.com/paper/1811.03509/) (2018)
- [What Makes Good In-context Demonstrations for Code Intelligence Tasks with LLMs?](https://mlsanity.com/paper/2304.07575/) (2023)
- [Log Parsing with Prompt-based Few-shot Learning](https://mlsanity.com/paper/2302.07435/) (2023)
- [UniParser: A Unified Log Parser for Heterogeneous Log Data](https://mlsanity.com/paper/2202.06569/) (2022)
- [A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair](https://mlsanity.com/paper/2304.08191/) (2023)
- [Self-Supervised Log Parsing](https://mlsanity.com/paper/2003.07905/) (2020)
- [Did We Miss Something Important? Studying and Exploring Variable-Aware Log Abstraction](https://mlsanity.com/paper/2304.11391/) (2023)
- [SemParser: A Semantic Parser for Log Analysis](https://mlsanity.com/paper/2112.12636/) (2021)
- [LogStamp: Automatic Online Log Parsing Based on Sequence Labelling](https://mlsanity.com/paper/2208.10282/) (2022)

## Built on by (most cited first, 16 of 16)

- [Characterization of Large Language Model Development in the Datacenter](https://mlsanity.com/paper/2403.07648/) (2024)
- [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025)
- [Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logs](https://mlsanity.com/paper/2404.00640/) (2024)
- [OptLLM: Optimal Assignment of Queries to Large Language Models](https://mlsanity.com/paper/2405.15130/) (2024)
- [Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge](https://mlsanity.com/paper/2412.01377/) (2024)
- [A Comparative Study on Large Language Models for Log Parsing](https://mlsanity.com/paper/2409.02474/) (2024)
- [Log Parsing using LLMs with Self-Generated In-Context Learning and Self-Correction](https://mlsanity.com/paper/2406.03376/) (2024)
- [LibreLog: Accurate and Efficient Unsupervised Log Parsing Using Open-Source Large Language Models](https://mlsanity.com/paper/2408.01585/) (2024)
- [System Log Parsing with Large Language Models: A Review](https://mlsanity.com/paper/2504.04877/) (2025)
- [HELP: Hierarchical Embeddings-based Log Parsing](https://mlsanity.com/paper/2408.08300/) (2024)
- [An Ethically Grounded LLM-Based Approach to Insider Threat Synthesis and Detection](https://mlsanity.com/paper/2509.06920/) (2025)
- [LLM-Powered Fully Automated Chaos Engineering: Towards Enabling Anyone to Build Resilient Software Systems at Low Cost](https://mlsanity.com/paper/2511.07865/) (2025)
- [SCOPE: Tree-based Self-Correcting Online Log Parsing via Syntactic-Semantic Collaboration](https://mlsanity.com/paper/2603.27247/) (2026)
- [Tabular PDF Information Extraction with Local LLMs and Layout-Aware Parsing: A Reliability Evaluation](https://mlsanity.com/paper/2604.00003/) (2026)
- [Traffic-Aware Randomized Smoothing for LLM-Based Network Intrusion Detection](https://mlsanity.com/paper/2607.13801/) (2026)
- [Legacy System Modernization with Coding Agents: A Case Study](https://mlsanity.com/paper/2608.28972/) (2026)

## Abstract (arXiv)

Software logs play an essential role in ensuring the reliability and maintainability of large-scale software systems, as they are often the sole source of runtime information. Log parsing, which converts raw log messages into structured data, is an important initial step towards downstream log analytics. In recent studies, ChatGPT, the current cutting-edge large language model (LLM), has been widely applied to a wide range of software engineering tasks. However, its performance in automated log parsing remains unclear. In this paper, we evaluate ChatGPT's ability to undertake log parsing by addressing two research questions. (1) Can ChatGPT effectively parse logs? (2) How does ChatGPT perform with different prompting methods? Our results show that ChatGPT can achieve promising results for log parsing with appropriate prompts, especially with few-shot prompting. Based on our findings, we outline several challenges and opportunities for ChatGPT-based log parsing.