# A Scenario-Oriented Benchmark for Assessing AIOps Algorithms in Microservice Management

arXiv: https://arxiv.org/abs/2407.14532 · HTML: https://mlsanity.com/paper/2407.14532/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 9 Jul 2024.

## Builds on (most cited first, 9 of 9)

- [TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis](https://mlsanity.com/paper/2210.02186/) (2022)
- [Log-based Anomaly Detection Without Log Parsing](https://mlsanity.com/paper/2108.01955/) (2021)
- [Anomaly Detection and Failure Root Cause Analysis in (Micro)Service-Based Cloud Applications: A Survey](https://mlsanity.com/paper/2105.12378/) (2021)
- [MicroHECL: High-Efficient Root Cause Localization in Large-Scale Microservice Systems](https://mlsanity.com/paper/2103.01782/) (2021)
- [CloudRCA: A Root Cause Analysis Framework for Cloud Computing Platforms](https://mlsanity.com/paper/2111.03753/) (2021)
- [Robust Failure Diagnosis of Microservice System through Multimodal Data](https://mlsanity.com/paper/2302.10512/) (2023)
- [TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly Detection Models](https://mlsanity.com/paper/2402.10802/) (2024)
- [Actionable and Interpretable Fault Localization for Recurring Failures in Online Service Systems](https://mlsanity.com/paper/2207.09021/) (2022)
- [On Root Cause Localization and Anomaly Mitigation through Causal Inference](https://mlsanity.com/paper/2212.04031/) (2022)

## Built on by (most cited first, 2 of 2)

- [Bridging Temporal and Textual Modalities: A Multimodal Framework for Automated Cloud Failure Root Cause Analysis](https://mlsanity.com/paper/2601.04709/) (2026)
- [STAR: A Stage-attributed Triage and Repair framework for RCA Agents in Microservices](https://mlsanity.com/paper/2605.15581/) (2026)

## Abstract (arXiv)

AIOps algorithms play a crucial role in the maintenance of microservice systems. Many previous benchmarks' performance leaderboard provides valuable guidance for selecting appropriate algorithms. However, existing AIOps benchmarks mainly utilize offline datasets to evaluate algorithms. They cannot consistently evaluate the performance of algorithms using real-time datasets, and the operation scenarios for evaluation are static, which is insufficient for effective algorithm selection. To address these issues, we propose an evaluation-consistent and scenario-oriented evaluation framework named MicroServo. The core idea is to build a live microservice benchmark to generate real-time datasets and consistently simulate the specific operation scenarios on it. MicroServo supports different leaderboards by selecting specific algorithms and datasets according to the operation scenarios. It also supports the deployment of various types of algorithms, enabling algorithms hot-plugging. At last, we test MicroServo with three typical microservice operation scenarios to demonstrate its efficiency and usability.