# Constructing Large-Scale Real-World Benchmark Datasets for AIOps

arXiv: https://arxiv.org/abs/2208.03938 · HTML: https://mlsanity.com/paper/2208.03938/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2022.

## Built on by (most cited first, 20 of 20)

- [ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning](https://mlsanity.com/paper/2412.03104/) (2024)
- [Large Language Models can Deliver Accurate and Interpretable Time Series Anomaly Detection](https://mlsanity.com/paper/2405.15370/) (2024)
- [Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspective](https://mlsanity.com/paper/2402.02820/) (2024)
- [Argos: Agentic Time-Series Anomaly Detection with Autonomous Rule Generation via Large Language Models](https://mlsanity.com/paper/2501.14170/) (2025)
- [AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges](https://mlsanity.com/paper/2304.04661/) (2023)
- [Can Multimodal LLMs Perform Time Series Anomaly Detection?](https://mlsanity.com/paper/2502.17812/) (2025)
- [LEMMA-RCA: A Large Multi-modal Multi-domain Dataset for Root Cause Analysis](https://mlsanity.com/paper/2406.05375/) (2024)
- [Graph of States: Solving Abductive Tasks with Large Language Models](https://mlsanity.com/paper/2603.21250/) (2026)
- [OpenRCA 2.0: From Outcome Labels to Causal Process Supervision](https://mlsanity.com/paper/2606.27154/) (2026)
- [Adaptive Thresholding Heuristic for KPI Anomaly Detection](https://mlsanity.com/paper/2308.10504/) (2023)
- [A Reliable Framework for Human-in-the-Loop Anomaly Detection in Time Series](https://mlsanity.com/paper/2405.03234/) (2024)
- [Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety](https://mlsanity.com/paper/2605.12729/) (2026)
- [Detecting Time Series Anomalies Like an Expert: A Multi-Agent LLM Framework with Specialized Analyzers](https://mlsanity.com/paper/2605.05725/) (2026)
- [hvEEGNet: exploiting hierarchical VAEs on EEG data for neuroscience applications](https://mlsanity.com/paper/2312.00799/) (2023)
- [ViTs: Teaching Machines to See Time Series Anomalies Like Human Experts](https://mlsanity.com/paper/2510.04710/) (2025)
- [LPCVAE: A Conditional VAE with Long-Term Dependency and Probabilistic Time-Frequency Fusion for Time Series Anomaly Detection](https://mlsanity.com/paper/2510.10915/) (2025)
- [Contextual and Seasonal LSTMs for Time Series Anomaly Detection](https://mlsanity.com/paper/2602.09690/) (2026)
- [LATS-RCA: Language Agent Tree Search for Root Cause Analysis in Microservices](https://mlsanity.com/paper/2605.03505/) (2026)
- [Anomaly Detection and Root Cause Analysis for Microservice Systems](https://mlsanity.com/paper/2606.09942/) (2026)
- [Optimizing Transformer Neural Network for Real-Time Outlier Detection on FPGAs](https://mlsanity.com/paper/2607.22786/) (2026)

## Abstract (arXiv)

Recently, AIOps (Artificial Intelligence for IT Operations) has been well studied in academia and industry to enable automated and effective software service management. Plenty of efforts have been dedicated to AIOps, including anomaly detection, root cause localization, incident management, etc. However, most existing works are evaluated on private datasets, so their generality and real performance cannot be guaranteed. The lack of public large-scale real-world datasets has prevented researchers and engineers from enhancing the development of AIOps. To tackle this dilemma, in this work, we introduce three public real-world, large-scale datasets about AIOps, mainly aiming at KPI anomaly detection, root cause localization on multi-dimensional data, and failure discovery and diagnosis. More importantly, we held three competitions in 2018/2019/2020 based on these datasets, attracting thousands of teams to participate. In the future, we will continue to publish more datasets and hold competitions to promote the development of AIOps further.