# Benchmarking Anomaly Detection Across Heterogeneous Cloud Telemetry Datasets

arXiv: https://arxiv.org/abs/2602.13288 · HTML: https://mlsanity.com/paper/2602.13288/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 47 references that are in the map.
- First posted on arXiv 7 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data](https://mlsanity.com/paper/2201.07284/) (2022)
- [Time-Series Anomaly Detection Service at Microsoft](https://mlsanity.com/paper/1906.03821/) (2019)
- [Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress](https://mlsanity.com/paper/2009.13807/) (2020)
- [TSMixer: Lightweight MLP-Mixer Model for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2306.09364/) (2023)
- [Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark](https://mlsanity.com/paper/1510.03336/) (2015)
- [Better than classical? The subtle art of benchmarking quantum machine learning models](https://mlsanity.com/paper/2403.07059/) (2024)
- [Extended Isolation Forest](https://mlsanity.com/paper/1811.02141/) (2018)
- [Precision and Recall for Time Series](https://mlsanity.com/paper/1803.03639/) (2018)
- [Exathlon: A Benchmark for Explainable Anomaly Detection over Time Series](https://mlsanity.com/paper/2010.05073/) (2020)
- [Anomaly Detection and Failure Root Cause Analysis in (Micro)Service-Based Cloud Applications: A Survey](https://mlsanity.com/paper/2105.12378/) (2021)
- [Anomaly Detection in a Large-scale Cloud Platform](https://mlsanity.com/paper/2010.10966/) (2020)
- [Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Dataset](https://mlsanity.com/paper/2411.09047/) (2024)
- [On Challenges of Cloud Monitoring](https://mlsanity.com/paper/1806.05914/) (2018)
- [Anomaly Detection in Cloud Components](https://mlsanity.com/paper/2005.08739/) (2020)

## Built on by (most cited first, 1 of 1)

- [ClouDens: Operational Context-Aware Anomaly Detection for Large-scale Cloud System Monitoring](https://mlsanity.com/paper/2607.18127/) (2026)

## Abstract (arXiv)

Anomaly detection is important for keeping cloud systems reliable and stable. Deep learning has improved time-series anomaly detection, but most models are evaluated on one dataset at a time. This raises questions about whether these models can handle different types of telemetry, especially in large-scale and high-dimensional environments. In this study, we evaluate four deep learning models, GRU, TCN, Transformer, and TSMixer. We also include Isolation Forest as a classical baseline. The models are tested across four telemetry datasets: the Numenta Anomaly Benchmark, Microsoft Cloud Monitoring dataset, Exathlon dataset, and IBM Console dataset. These datasets differ in structure, dimensionality, and labelling strategy. They include univariate time series, synthetic multivariate workloads, and real-world production telemetry with over 100,000 features. We use a unified training and evaluation pipeline across all datasets. The evaluation includes NAB-style metrics to capture early detection behaviour for datasets where anomalies persist over contiguous time intervals. This enables window-based scoring in settings where anomalies occur over contiguous time intervals, even when labels are recorded at the point level. The unified setup enables consistent analysis of model behaviour under shared scoring and calibration assumptions. Our results demonstrate that anomaly detection performance in cloud systems is governed not only by model architecture, but critically by calibration stability and feature-space geometry. By releasing our preprocessing pipelines, benchmark configuration, and evaluation artifacts, we aim to support reproducible and deployment-aware evaluation of anomaly detection systems for cloud environments.