# Face It Yourselves: An LLM-Based Two-Stage Strategy to Localize Configuration Errors via Logs

arXiv: https://arxiv.org/abs/2404.00640 · HTML: https://mlsanity.com/paper/2404.00640/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 58 references that are in the map.
- First posted on arXiv 31 Mar 2024.

## Builds on (most cited first, 9 of 9)

- [Automatic Root Cause Analysis via Large Language Models for Cloud Incidents](https://mlsanity.com/paper/2305.15778/) (2023)
- [Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics](https://mlsanity.com/paper/2008.06448/) (2020)
- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [A Survey on Automated Log Analysis for Reliability Engineering](https://mlsanity.com/paper/2009.07237/) (2020)
- [SemParser: A Semantic Parser for Log Analysis](https://mlsanity.com/paper/2112.12636/) (2021)
- [Log Parsing: How Far Can ChatGPT Go?](https://mlsanity.com/paper/2306.01590/) (2023)
- [Exploring the Effectiveness of LLMs in Automated Logging Generation: An Empirical Study](https://mlsanity.com/paper/2307.05950/) (2023)
- [EvLog: Identifying Anomalous Logs over Software Evolution](https://mlsanity.com/paper/2306.01509/) (2023)
- [ConfInLog: Leveraging Software Logs to Infer Configuration Constraints](https://mlsanity.com/paper/2103.11561/) (2021)

## Built on by (most cited first, 17 of 17)

- [Large Language Models for Cyber Security: A Systematic Literature Review](https://mlsanity.com/paper/2405.04760/) (2024)
- [Frontier AI's Impact on the Cybersecurity Landscape](https://mlsanity.com/paper/2504.05408/) (2025)
- [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025)
- [LogEval: A Comprehensive Benchmark Suite for Large Language Models In Log Analysis](https://mlsanity.com/paper/2407.01896/) (2024)
- [LEMMA-RCA: A Large Multi-modal Multi-domain Dataset for Root Cause Analysis](https://mlsanity.com/paper/2406.05375/) (2024)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [Agentic Memory Enhanced Recursive Reasoning for Root Cause Localization in Microservices](https://mlsanity.com/paper/2601.02732/) (2026)
- [Hypothesize-Then-Verify: Speculative Root Cause Analysis for Microservices with Pathwise Parallelism](https://mlsanity.com/paper/2601.02736/) (2026)
- [Simplifying Root Cause Analysis in Kubernetes with StateGraph and LLM](https://mlsanity.com/paper/2506.02490/) (2025)
- [Foundation Models for Anomaly Detection: Vision and Challenges](https://mlsanity.com/paper/2502.06911/) (2025)
- [Identifying Performance-Sensitive Configurations in Software Systems through Code Analysis with LLM Agents](https://mlsanity.com/paper/2406.12806/) (2024)
- [SDLog: A Deep Learning Framework for Detecting Sensitive Information in Software Logs](https://mlsanity.com/paper/2505.14976/) (2025)
- [Online Multi-modal Root Cause Identification in Microservice Systems](https://mlsanity.com/paper/2410.10021/) (2024)
- [A Blueprint for AI-Driven Software Quality: Integrating LLMs with Established Standards](https://mlsanity.com/paper/2505.13766/) (2025)
- [Towards In-Depth Root Cause Localization for Microservices with Multi-Agent Recursion-of-Thought](https://mlsanity.com/paper/2605.14866/) (2026)
- [Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniques](https://mlsanity.com/paper/2507.13629/) (2025)
- [Instruction-Tuned LLMs for Parsing and Mining Unstructured Logs on Leadership HPC Systems](https://mlsanity.com/paper/2604.05168/) (2026)

## Abstract (arXiv)

Configurable software systems are prone to configuration errors, resulting in significant losses to companies. However, diagnosing these errors is challenging due to the vast and complex configuration space. These errors pose significant challenges for both experienced maintainers and new end-users, particularly those without access to the source code of the software systems. Given that logs are easily accessible to most end-users, we conduct a preliminary study to outline the challenges and opportunities of utilizing logs in localizing configuration errors. Based on the insights gained from the preliminary study, we propose an LLM-based two-stage strategy for end-users to localize the root-cause configuration properties based on logs. We further implement a tool, LogConfigLocalizer, aligned with the design of the aforementioned strategy, hoping to assist end-users in coping with configuration errors through log analysis. To the best of our knowledge, this is the first work to localize the root-cause configuration properties for end-users based on Large Language Models~(LLMs) and logs. We evaluate the proposed strategy on Hadoop by LogConfigLocalizer and prove its efficiency with an average accuracy as high as 99.91%. Additionally, we also demonstrate the effectiveness and necessity of different phases of the methodology by comparing it with two other variants and a baseline tool. Moreover, we validate the proposed methodology through a practical case study to demonstrate its effectiveness and feasibility.