# Knowledge-aware Alert Aggregation in Large-scale Cloud Systems: a Hybrid Approach

arXiv: https://arxiv.org/abs/2403.06485 · HTML: https://mlsanity.com/paper/2403.06485/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 19 of 19)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://mlsanity.com/paper/2110.07602/) (2021)
- [Towards Reasoning in Large Language Models: A Survey](https://mlsanity.com/paper/2212.10403/) (2022)
- [Automatic Root Cause Analysis via Large Language Models for Cloud Incidents](https://mlsanity.com/paper/2305.15778/) (2023)
- [Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks](https://mlsanity.com/paper/2102.02017/) (2021)
- [Towards an Understanding of Large Language Models in Software Engineering Tasks](https://mlsanity.com/paper/2308.11396/) (2023)
- [Revisiting, Benchmarking and Exploring API Recommendation: How Far Are We?](https://mlsanity.com/paper/2112.12653/) (2021)
- [Using Deep Learning to Generate Complete Log Statements](https://mlsanity.com/paper/2201.04837/) (2022)
- [Groot: An Event-graph-based Approach for Root Cause Analysis in Industrial Settings](https://mlsanity.com/paper/2108.00344/) (2021)
- [Logzip: Extracting Hidden Structures via Iterative Clustering for Log Compression](https://mlsanity.com/paper/1910.00409/) (2019)
- [AutoTSG: Learning and Synthesis for Incident Troubleshooting](https://mlsanity.com/paper/2205.13457/) (2022)
- [Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attention](https://mlsanity.com/paper/2302.06914/) (2023)
- [Incident-aware Duplicate Ticket Aggregation for Cloud Systems](https://mlsanity.com/paper/2302.09520/) (2023)
- [Identifying Performance Issues in Cloud Service Systems Based on Relational-Temporal Features](https://mlsanity.com/paper/2307.10869/) (2023)
- [Characterizing and Mitigating Anti-patterns of Alerts in Industrial Cloud Systems](https://mlsanity.com/paper/2204.09670/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Survey of AIOps in the Era of Large Language Models](https://mlsanity.com/paper/2507.12472/) (2025)
- [Demystifying and Extracting Fault-indicating Information from Logs for Failure Diagnosis](https://mlsanity.com/paper/2409.13561/) (2024)
- [GALA: Can Graph-Augmented Large Language Model Agentic Workflows Elevate Root Cause Analysis?](https://mlsanity.com/paper/2508.12472/) (2025)
- [SRE-Marathon: A Continuous, Change-Driven Benchmark for Autonomous Site Reliability Agents](https://mlsanity.com/paper/2609.33023/) (2026)

## Abstract (arXiv)

Due to the scale and complexity of cloud systems, a system failure would trigger an "alert storm", i.e., massive correlated alerts. Although these alerts can be traced back to a few root causes, the overwhelming number makes it infeasible for manual handling. Alert aggregation is thus critical to help engineers concentrate on the root cause and facilitate failure resolution. Existing methods typically utilize semantic similarity-based methods or statistical methods to aggregate alerts. However, semantic similarity-based methods overlook the causal rationale of alerts, while statistical methods can hardly handle infrequent alerts. To tackle these limitations, we introduce leveraging external knowledge, i.e., Standard Operation Procedure (SOP) of alerts as a supplement. We propose COLA, a novel hybrid approach based on correlation mining and LLM (Large Language Model) reasoning for online alert aggregation. The correlation mining module effectively captures the temporal and spatial relations between alerts, measuring their correlations in an efficient manner. Subsequently, only uncertain pairs with low confidence are forwarded to the LLM reasoning module for detailed analysis. This hybrid design harnesses both statistical evidence for frequent alerts and the reasoning capabilities of computationally intensive LLMs, ensuring the overall efficiency of COLA in handling large volumes of alerts in practical scenarios. We evaluate COLA on three datasets collected from the production environment of a large-scale cloud platform. The experimental results show COLA achieves F1-scores from 0.901 to 0.930, outperforming state-of-the-art methods and achieving comparable efficiency. We also share our experience in deploying COLA in our real-world cloud system, Cloud X.