# Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4

arXiv: https://arxiv.org/abs/2401.13810 · HTML: https://mlsanity.com/paper/2401.13810/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 51 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Software Testing with Large Language Models: Survey, Landscape, and Vision](https://mlsanity.com/paper/2307.07221/) (2023)
- [Jigsaw: Large Language Models meet Program Synthesis](https://mlsanity.com/paper/2112.02969/) (2021)
- [Automatic Root Cause Analysis via Large Language Models for Cloud Incidents](https://mlsanity.com/paper/2305.15778/) (2023)
- [Causal Inference-Based Root Cause Analysis for Online Service Systems with Intervention Recognition](https://mlsanity.com/paper/2206.05871/) (2022)
- [NUBIA: NeUral Based Interchangeability Assessor for Text Generation](https://mlsanity.com/paper/2004.14667/) (2020)
- [Robust Failure Diagnosis of Microservice System through Multimodal Data](https://mlsanity.com/paper/2302.10512/) (2023)
- [Using Deep Learning to Generate Complete Log Statements](https://mlsanity.com/paper/2201.04837/) (2022)
- [DeCaf: Diagnosing and Triaging Performance Issues in Large-Scale Cloud Services](https://mlsanity.com/paper/1910.05339/) (2019)
- [Picking Pearl From Seabed: Extracting Artefacts from Noisy Issue Triaging Collaborative Conversations for Hybrid Cloud Services](https://mlsanity.com/paper/2105.15065/) (2021)

## Built on by (most cited first, 28 of 28)

- [AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds](https://mlsanity.com/paper/2501.06706/) (2025)
- [TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning](https://mlsanity.com/paper/2409.13035/) (2024)
- [X-lifecycle Learning for Cloud Incident Management using LLMs](https://mlsanity.com/paper/2404.03662/) (2024)
- [Building AI Agents for Autonomous Clouds: Challenges and Design Principles](https://mlsanity.com/paper/2407.12165/) (2024)
- [AgentFM: Role-Aware Failure Management for Distributed Databases with LLM-Driven Multi-Agents](https://mlsanity.com/paper/2504.06614/) (2025)
- [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025)
- [Large Language Models for Security Operations Centers: A Comprehensive Survey](https://mlsanity.com/paper/2509.10858/) (2025)
- [Agentic Memory Enhanced Recursive Reasoning for Root Cause Localization in Microservices](https://mlsanity.com/paper/2601.02732/) (2026)
- [Hypothesize-Then-Verify: Speculative Root Cause Analysis for Microservices with Pathwise Parallelism](https://mlsanity.com/paper/2601.02736/) (2026)
- [The Multi-Agent Fault Localization System Based on Monte Carlo Tree Search Approach](https://mlsanity.com/paper/2507.22800/) (2025)
- [OpsAgent: An Evolving Multi-agent System for Incident Management in Microservices](https://mlsanity.com/paper/2510.24145/) (2025)
- [GALA: Can Graph-Augmented Large Language Model Agentic Workflows Elevate Root Cause Analysis?](https://mlsanity.com/paper/2508.12472/) (2025)
- [Simplifying Root Cause Analysis in Kubernetes with StateGraph and LLM](https://mlsanity.com/paper/2506.02490/) (2025)
- [Debugging the Debuggers: Failure-Anchored Structured Recovery for Software Engineering Agents](https://mlsanity.com/paper/2605.08717/) (2026)
- [Generative transformations and patterns in LLM-native approaches for software verification and falsification](https://mlsanity.com/paper/2404.09384/) (2024)
- [Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety](https://mlsanity.com/paper/2605.12729/) (2026)
- [ENCO: Life-Cycle Management of Enterprise-Grade Copilots](https://mlsanity.com/paper/2412.06099/) (2024)
- [Prompting for Performance: Exploring LLMs for Configuring Software](https://mlsanity.com/paper/2507.09790/) (2025)
- [Bridging Temporal and Textual Modalities: A Multimodal Framework for Automated Cloud Failure Root Cause Analysis](https://mlsanity.com/paper/2601.04709/) (2026)
- [SiriusHelper: An LLM Agent-Based Operations Assistant for Big Data Platforms](https://mlsanity.com/paper/2605.00043/) (2026)
- [TriCalRAG: A Three-Strategy, Retrieval-Augmented Benchmark for On-Premise LLM-Based Root Cause Analysis in AIOps](https://mlsanity.com/paper/2609.14762/) (2026)
- [A Comparative Study of Recent Large Language Models on Generating Hospital Discharge Summaries for Lung Cancer Patients](https://mlsanity.com/paper/2411.03805/) (2024)
- [LLM-Powered Fully Automated Chaos Engineering: Towards Enabling Anyone to Build Resilient Software Systems at Low Cost](https://mlsanity.com/paper/2511.07865/) (2025)
- [OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learning](https://mlsanity.com/paper/2605.02906/) (2026)
- [FixItFlow: Automated Troubleshooting Guide Generation from Cloud Incidents](https://mlsanity.com/paper/2607.13035/) (2026)
- [Beyond Fault Localization: A Trajectory-Level Study of LLM Agents for Microservice Root Cause Analysis](https://mlsanity.com/paper/2608.21310/) (2026)
- [From General Agents to RCA Experts: A Self-Evolving Harness for Root Cause Analysis](https://mlsanity.com/paper/2608.25661/) (2026)
- [TriFleetRCA: On-Premise LLM Root Cause Analysis for Kubernetes](https://mlsanity.com/paper/2609.23766/) (2026)

## Abstract (arXiv)

Root Cause Analysis (RCA) plays a pivotal role in the incident diagnosis process for cloud services, requiring on-call engineers to identify the primary issues and implement corrective actions to prevent future recurrences. Improving the incident RCA process is vital for minimizing service downtime, customer impact and manual toil. Recent advances in artificial intelligence have introduced state-of-the-art Large Language Models (LLMs) like GPT-4, which have proven effective in tackling various AIOps problems, ranging from code authoring to incident management. Nonetheless, the GPT-4 model's immense size presents challenges when trying to fine-tune it on user data because of the significant GPU resource demand and the necessity for continuous model fine-tuning with the emergence of new data. To address the high cost of fine-tuning LLM, we propose an in-context learning approach for automated root causing, which eliminates the need for fine-tuning. We conduct extensive study over 100,000 production incidents, comparing several large language models using multiple metrics. The results reveal that our in-context learning approach outperforms the previous fine-tuned large language models such as GPT-3 by an average of 24.8% across all metrics, with an impressive 49.7% improvement over the zero-shot model. Moreover, human evaluation involving actual incident owners demonstrates its superiority over the fine-tuned model, achieving a 43.5% improvement in correctness and an 8.7% enhancement in readability. The impressive results demonstrate the viability of utilizing a vanilla GPT model for the RCA task, thereby avoiding the high computational and maintenance costs associated with a fine-tuned model.