# Interpretable Online Log Analysis Using Large Language Models with Prompt Strategies

arXiv: https://arxiv.org/abs/2308.07610 · HTML: https://mlsanity.com/paper/2308.07610/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 15 Aug 2023.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [Log-based Anomaly Detection with Deep Learning: How Far Are We?](https://mlsanity.com/paper/2202.04301/) (2022)
- [Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection](https://mlsanity.com/paper/2107.05908/) (2021)
- [Did We Miss Something Important? Studying and Exploring Variable-Aware Log Abstraction](https://mlsanity.com/paper/2304.11391/) (2023)
- [SemParser: A Semantic Parser for Log Analysis](https://mlsanity.com/paper/2112.12636/) (2021)
- [LogStamp: Automatic Online Log Parsing Based on Sequence Labelling](https://mlsanity.com/paper/2208.10282/) (2022)

## Built on by (most cited first, 30 of 38)

- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [When LLMs Meet Cybersecurity: A Systematic Literature Review](https://mlsanity.com/paper/2405.03644/) (2024)
- [Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code](https://mlsanity.com/paper/2311.07989/) (2023)
- [AD-LLM: Benchmarking Large Language Models for Anomaly Detection](https://mlsanity.com/paper/2412.11142/) (2024)
- [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025)
- [LogLM: From Task-based to Instruction-based Automated Log Analysis](https://mlsanity.com/paper/2410.09352/) (2024)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge](https://mlsanity.com/paper/2412.01377/) (2024)
- [Log Anomaly Detection with Large Language Models via Knowledge-Enriched Fusion](https://mlsanity.com/paper/2512.11997/) (2025)
- [RedChronos: A Large Language Model-Based Log Analysis System for Insider Threat Detection in Enterprises](https://mlsanity.com/paper/2503.02702/) (2025)
- [A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data](https://mlsanity.com/paper/2601.17717/) (2026)
- [A Comparative Study on Large Language Models for Log Parsing](https://mlsanity.com/paper/2409.02474/) (2024)
- [Generative AI for Self-Adaptive Systems: State of the Art and Research Roadmap](https://mlsanity.com/paper/2512.04680/) (2025)
- [Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidance](https://mlsanity.com/paper/2408.12910/) (2024)
- [AdaptiveLog: An Adaptive Log Analysis Framework with the Collaboration of Large and Small Language Model](https://mlsanity.com/paper/2501.11031/) (2025)
- [RulePilot: An LLM-Powered Agent for Security Rule Generation](https://mlsanity.com/paper/2511.12224/) (2025)
- [E2E-REME: Towards End-to-End Microservices Auto-Remediation via Experience-Simulation Reinforcement Fine-Tuning](https://mlsanity.com/paper/2604.11094/) (2026)
- [System Log Parsing with Large Language Models: A Review](https://mlsanity.com/paper/2504.04877/) (2025)
- [LLMLogAnalyzer: A Clustering-Based Log Analysis Chatbot using Large Language Models](https://mlsanity.com/paper/2510.24031/) (2025)
- [Tracing the Data Trail: A Survey of Data Provenance, Transparency and Traceability in LLMs](https://mlsanity.com/paper/2601.14311/) (2026)
- [Team UTSA-NLP at SemEval 2024 Task 5: Prompt Ensembling for Argument Reasoning in Civil Procedures with GPT4](https://mlsanity.com/paper/2404.01961/) (2024)
- [RuleGenie: SIEM Detection Rule Set Optimization](https://mlsanity.com/paper/2505.06701/) (2025)
- [CAM-LDS: Cyber Attack Manifestations for Automatic Interpretation of System Logs and Security Alerts](https://mlsanity.com/paper/2603.04186/) (2026)
- [DMFI: A Dual-Modality Log Analysis Framework for Insider Threat Detection with LoRA-Tuned Language Models](https://mlsanity.com/paper/2508.05694/) (2025)
- [An Ethically Grounded LLM-Based Approach to Insider Threat Synthesis and Detection](https://mlsanity.com/paper/2509.06920/) (2025)
- [From Flows to Words: Can Zero-/Few-Shot LLMs Detect Network Intrusions? A Grammar-Constrained, Calibrated Evaluation on UNSW-NB15](https://mlsanity.com/paper/2510.17883/) (2025)
- [Log2Sig: Frequency-Aware Insider Threat Detection via Multivariate Behavioral Signal Decomposition](https://mlsanity.com/paper/2508.05696/) (2025)
- [RationAnomaly: Log Anomaly Detection with Rationality via Chain-of-Thought and Reinforcement Learning](https://mlsanity.com/paper/2509.14693/) (2025)
- [EPIC: Generative AI Platform for Accelerating HPC Operational Data Analytics](https://mlsanity.com/paper/2509.16212/) (2025)
- [Scalable and Efficient Large-Scale Log Analysis with LLMs: An IT Software Support Case Study](https://mlsanity.com/paper/2511.14803/) (2025)

## Abstract (arXiv)

Automated log analysis is crucial in modern software-intensive systems for facilitating program comprehension throughout software maintenance and engineering life cycles. Existing methods perform tasks such as log parsing and log anomaly detection by providing a single prediction value without interpretation. However, given the increasing volume of system events, the limited interpretability of analysis results hinders analysts' comprehension of program status and their ability to take appropriate actions. Moreover, these methods require substantial in-domain training data, and their performance declines sharply (by up to 62.5%) in online scenarios involving unseen logs from new domains, a common occurrence due to rapid software updates. In this paper, we propose LogPrompt, a novel interpretable log analysis approach for online scenarios. LogPrompt employs large language models (LLMs) to perform online log analysis tasks via a suite of advanced prompt strategies tailored for log tasks, which enhances LLMs' performance by up to 380.7% compared with simple prompts. Experiments on nine publicly available evaluation datasets across two tasks demonstrate that LogPrompt, despite requiring no in-domain training, outperforms existing approaches trained on thousands of logs by up to 55.9%. We also conduct a human evaluation of LogPrompt's interpretability, with six practitioners possessing over 10 years of experience, who highly rated the generated content in terms of usefulness and readability (averagely 4.42/5). LogPrompt also exhibits remarkable compatibility with open-source and smaller-scale LLMs, making it flexible for practical deployment. Code of LogPrompt is available at https://github.com/lunyiliu/LogPrompt.