# Assess and Summarize: Improve Outage Understanding with Large Language Models

arXiv: https://arxiv.org/abs/2305.18084 · HTML: https://mlsanity.com/paper/2305.18084/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2023.

## Built on by (most cited first, 17 of 17)

- [UFO: A UI-Focused Agent for Windows OS Interaction](https://mlsanity.com/paper/2402.07939/) (2024)
- [RCAgent: Cloud Root Cause Analysis by Autonomous Agents with Tool-Augmented Large Language Models](https://mlsanity.com/paper/2310.16340/) (2023)
- [Xpert: Empowering Incident Management with Query Recommendations via Large Language Models](https://mlsanity.com/paper/2312.11988/) (2023)
- [Building AI Agents for Autonomous Clouds: Challenges and Design Principles](https://mlsanity.com/paper/2407.12165/) (2024)
- [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025)
- [LLMs with Industrial Lens: Deciphering the Challenges and Prospects -- A Survey](https://mlsanity.com/paper/2402.14558/) (2024)
- [The Multi-Agent Fault Localization System Based on Monte Carlo Tree Search Approach](https://mlsanity.com/paper/2507.22800/) (2025)
- [StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis](https://mlsanity.com/paper/2510.10074/) (2025)
- [No Size Fits All: The Perils and Pitfalls of Leveraging LLMs Vary with Company Size](https://mlsanity.com/paper/2408.01444/) (2024)
- [PACE-LM: Prompting and Augmentation for Calibrated Confidence Estimation with GPT-4 in Cloud Incident Root Cause Analysis](https://mlsanity.com/paper/2309.05833/) (2023)
- [PRAXIS: Integrating Program Analysis with Observability for Root-Cause Analysis](https://mlsanity.com/paper/2512.22113/) (2025)
- [Enhanced Fine-Tuning of Lightweight Domain-Specific Q&A Model Based on Large Language Models](https://mlsanity.com/paper/2408.12247/) (2024)
- [InfraBench: Evaluating Infrastructure Agents Across Layers, Lifecycle, and Risk](https://mlsanity.com/paper/2608.11234/) (2026)
- [Evaluation of a Foundational Model and Stochastic Models for Forecasting Sporadic or Spiky Production Outages of High-Performance Machine Learning Services](https://mlsanity.com/paper/2507.01067/) (2025)
- [LLM-Augmented Knowledge Base Construction For Root Cause Analysis](https://mlsanity.com/paper/2604.06171/) (2026)
- [OpsLLM: Construction of Large Language Model for Software Operations with Multi-stage Learning](https://mlsanity.com/paper/2605.02906/) (2026)
- [From General Agents to RCA Experts: A Self-Evolving Harness for Root Cause Analysis](https://mlsanity.com/paper/2608.25661/) (2026)

## Abstract (arXiv)

Cloud systems have become increasingly popular in recent years due to their flexibility and scalability. Each time cloud computing applications and services hosted on the cloud are affected by a cloud outage, users can experience slow response times, connection issues or total service disruption, resulting in a significant negative business impact. Outages are usually comprised of several concurring events/source causes, and therefore understanding the context of outages is a very challenging yet crucial first step toward mitigating and resolving outages. In current practice, on-call engineers with in-depth domain knowledge, have to manually assess and summarize outages when they happen, which is time-consuming and labor-intensive. In this paper, we first present a large-scale empirical study investigating the way on-call engineers currently deal with cloud outages at Microsoft, and then present and empirically validate a novel approach (dubbed Oasis) to help the engineers in this task. Oasis is able to automatically assess the impact scope of outages as well as to produce human-readable summarization. Specifically, Oasis first assesses the impact scope of an outage by aggregating relevant incidents via multiple techniques. Then, it generates a human-readable summary by leveraging fine-tuned large language models like GPT-3.x. The impact assessment component of Oasis was introduced in Microsoft over three years ago, and it is now widely adopted, while the outage summarization component has been recently introduced, and in this article we present the results of an empirical evaluation we carried out on 18 real-world cloud systems as well as a human-based evaluation with outage owners. The results show that Oasis can effectively and efficiently summarize outages, and lead Microsoft to deploy its first prototype which is currently under experimental adoption by some of the incident teams.