# Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark

arXiv: https://arxiv.org/abs/1510.03336 · HTML: https://mlsanity.com/paper/1510.03336/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2015.

## Built on by (most cited first, 30 of 88)

- [ADBench: Anomaly Detection Benchmark](https://mlsanity.com/paper/2206.09426/) (2022)
- [Deep Learning for Time Series Anomaly Detection: A Survey](https://mlsanity.com/paper/2211.05244/) (2022)
- [Local Evaluation of Time Series Anomaly Detection Algorithms](https://mlsanity.com/paper/2206.13167/) (2022)
- [TAB: Unified Benchmarking of Time Series Anomaly Detection Methods](https://mlsanity.com/paper/2506.18046/) (2025)
- [Neural Contextual Anomaly Detection for Time Series](https://mlsanity.com/paper/2107.07702/) (2021)
- [Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advances](https://mlsanity.com/paper/2409.11884/) (2024)
- [Merlion: A Machine Learning Library for Time Series](https://mlsanity.com/paper/2109.09265/) (2021)
- [Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series](https://mlsanity.com/paper/2303.01272/) (2023)
- [Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspective](https://mlsanity.com/paper/2402.02820/) (2024)
- [Sintel: A Machine Learning Framework to Extract Insights from Signals](https://mlsanity.com/paper/2204.09108/) (2022)
- [Are we certain it's anomalous?](https://mlsanity.com/paper/2211.09224/) (2022)
- [A Comparative Study on Unsupervised Anomaly Detection for Time Series: Experiments and Analysis](https://mlsanity.com/paper/2209.04635/) (2022)
- [Harnessing Vision-Language Models for Time Series Anomaly Detection](https://mlsanity.com/paper/2506.06836/) (2025)
- [Deep Contrastive One-Class Time Series Anomaly Detection](https://mlsanity.com/paper/2207.01472/) (2022)
- [Is it worth it? Comparing six deep and classical methods for unsupervised anomaly detection in time series](https://mlsanity.com/paper/2212.11080/) (2022)
- [How Far Should We Look Back to Achieve Effective Real-Time Time-Series Anomaly Detection?](https://mlsanity.com/paper/2102.06560/) (2021)
- [SALAD: Self-Adaptive Lightweight Anomaly Detection for Real-time Recurrent Time Series](https://mlsanity.com/paper/2104.09968/) (2021)
- [A Learning-Based Fast Uplink Grant for Massive IoT via Support Vector Machines and Long Short-Term Memory](https://mlsanity.com/paper/2108.10070/) (2021)
- [Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Dataset](https://mlsanity.com/paper/2411.09047/) (2024)
- [mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scale](https://mlsanity.com/paper/2506.21550/) (2025)
- [A Modular and Unified Framework for Detecting and Localizing Video Anomalies](https://mlsanity.com/paper/2103.11299/) (2021)
- [MOSPAT: AutoML based Model Selection and Parameter Tuning for Time Series Anomaly Detection](https://mlsanity.com/paper/2205.11755/) (2022)
- [RePAD2: Real-Time, Lightweight, and Adaptive Anomaly Detection for Open-Ended Time Series](https://mlsanity.com/paper/2303.00409/) (2023)
- [Artificial Intelligence-Based Multiscale Temporal Modeling for Anomaly Detection in Cloud Services](https://mlsanity.com/paper/2508.14503/) (2025)
- [ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Response](https://mlsanity.com/paper/2604.21199/) (2026)
- [Data-driven Design of Context-aware Monitors for Hazard Prediction in Artificial Pancreas Systems](https://mlsanity.com/paper/2104.02545/) (2021)
- [Anomaly Detection in Predictive Maintenance: A New Evaluation Framework for Temporal Unsupervised Anomaly Detection Algorithms](https://mlsanity.com/paper/2105.12818/) (2021)
- [Open Challenges in Time Series Anomaly Detection: An Industry Perspective](https://mlsanity.com/paper/2502.05392/) (2025)
- [TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoning](https://mlsanity.com/paper/2606.01498/) (2026)
- [Stacked Residuals of Dynamic Layers for Time Series Anomaly Detection](https://mlsanity.com/paper/2202.12457/) (2022)

## Abstract (arXiv)

Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations; examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment of open-source tools to test and measure anomaly detection algorithms on streaming data. The perfect detector would detect all anomalies as soon as possible, trigger no false alarms, work with real-world time-series data across a variety of domains, and automatically adapt to changing statistics. Rewarding these characteristics is formalized in NAB, using a scoring algorithm designed for streaming data. NAB evaluates detectors on a benchmark dataset with labeled, real-world time-series data. We present these components, and give results and analyses for several open source, commercially-used algorithms. The goal for NAB is to provide a standard, open source framework with which the research community can compare and evaluate different algorithms for detecting anomalies in streaming data.