mlsanityData to 7 Oct 2026

Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark

arXiv 1510.03336 ↗First posted 12 Oct 2015cs.AI cs.LGcited by 87

Not on the map. Why?

Abstract (arXiv)

Much of the world's data is streaming, time-series data, where anomalies give significant information in critical situations; examples abound in domains such as finance, IT, security, medical, and energy. Yet detecting anomalies in streaming data is a difficult task, requiring detectors to process data in real-time, not batches, and learn while simultaneously making predictions. There are no benchmarks to adequately test and score the efficacy of real-time anomaly detectors. Here we propose the Numenta Anomaly Benchmark (NAB), which attempts to provide a controlled and repeatable environment of open-source tools to test and measure anomaly detection algorithms on streaming data. The perfect detector would detect all anomalies as soon as possible, trigger no false alarms, work with real-world time-series data across a variety of domains, and automatically adapt to changing statistics. Rewarding these characteristics is formalized in NAB, using a scoring algorithm designed for streaming data. NAB evaluates detectors on a benchmark dataset with labeled, real-world time-series data. We present these components, and give results and analyses for several open source, commercially-used algorithms. The goal for NAB is to provide a standard, open source framework with which the research community can compare and evaluate different algorithms for detecting anomalies in streaming data.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Oct 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 14; 2023: 12; 2024: 11; 2025: 20; 2026: 17.

87 papers.

Sort:

By area:

  1. 2022ADBench: Anomaly Detection Benchmarkcited by 230
  2. 2022Deep Learning for Time Series Anomaly Detection: A Surveycited by 130
  3. 2022Local Evaluation of Time Series Anomaly Detection Algorithmscited by 79
  4. 2025TAB: Unified Benchmarking of Time Series Anomaly Detection Methodscited by 58
  5. 2021Neural Contextual Anomaly Detection for Time Seriescited by 42
  6. 2021Merlion: A Machine Learning Library for Time Seriescited by 25
  7. 2023Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Seriescited by 24
  8. 2024Out-of-Distribution Detection: A Task-Oriented Survey of Recent Advancescited by 24
  9. 2024Revisiting VAE for Unsupervised Time Series Anomaly Detection: A Frequency Perspectivecited by 22
  10. 2022Sintel: A Machine Learning Framework to Extract Insights from Signalscited by 16
  11. 2022Are we certain it's anomalous?cited by 13
  12. 2022Deep Contrastive One-Class Time Series Anomaly Detectioncited by 10
38 more
  1. 2022A Comparative Study on Unsupervised Anomaly Detection for Time Series: Experiments and Analysiscited by 10
  2. 2022Is it worth it? Comparing six deep and classical methods for unsupervised anomaly detection in time seriescited by 9
  3. 2025Harnessing Vision-Language Models for Time Series Anomaly Detectioncited by 9
  4. 2021How Far Should We Look Back to Achieve Effective Real-Time Time-Series Anomaly Detection?cited by 8
  5. 2021SALAD: Self-Adaptive Lightweight Anomaly Detection for Real-time Recurrent Time Seriescited by 8
  6. 2021A Learning-Based Fast Uplink Grant for Massive IoT via Support Vector Machines and Long Short-Term Memorycited by 7
  7. 2021A Modular and Unified Framework for Detecting and Localizing Video Anomaliescited by 6
  8. 2023RePAD2: Real-Time, Lightweight, and Adaptive Anomaly Detection for Open-Ended Time Seriescited by 6
  9. 2024Anomaly Detection in Large-Scale Cloud Systems: An Industry Case and Datasetcited by 6
  10. 2025mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scalecited by 6
  11. 2025Artificial Intelligence-Based Multiscale Temporal Modeling for Anomaly Detection in Cloud Servicescited by 6
  12. 2021Data-driven Design of Context-aware Monitors for Hazard Prediction in Artificial Pancreas Systemscited by 5
  13. 2021Anomaly Detection in Predictive Maintenance: A New Evaluation Framework for Temporal Unsupervised Anomaly Detection Algorithmscited by 5
  14. 2022MOSPAT: AutoML based Model Selection and Parameter Tuning for Time Series Anomaly Detectioncited by 5
  15. 2025Open Challenges in Time Series Anomaly Detection: An Industry Perspectivecited by 5
  16. 2026ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Responsecited by 5
  17. 2026TimeSage-MT: A Multi-Turn Benchmark for Evaluating Agentic Time Series Reasoningcited by 5
  18. 2023ALGAN: Time Series Anomaly Detection with Adjusted-LSTM GANcited by 4
  19. 2025RoCA: Robust Contrastive One-class Time Series Anomaly Detection with Contaminated Datacited by 4
  20. 2022Stacked Residuals of Dynamic Layers for Time Series Anomaly Detectioncited by 3
  21. 2023Adaptive Thresholding Heuristic for KPI Anomaly Detectioncited by 3
  22. 2024I/O in Machine Learning Applications on HPC Systems: A 360-degree Surveycited by 3
  23. 2024A Reliable Framework for Human-in-the-Loop Anomaly Detection in Time Seriescited by 3
  24. 2024Anomaly Prediction: A Novel Approach with Explicit Delay and Horizoncited by 3
  25. 2025GAL-MAD: Towards Explainable Anomaly Detection in Microservice Applications Using Graph Attention Networkscited by 3
  26. 2021NVAE-GAN Based Approach for Unsupervised Time Series Anomaly Detectioncited by 2
  27. 2021Distribution Agnostic Symbolic Representations for Time Series Dimensionality Reduction and Online Anomaly Detectioncited by 2
  28. 2022Hybridization of Capsule and LSTM Networks for unsupervised anomaly detection on multivariate datacited by 2
  29. 2022Latent Neural Stochastic Differential Equations for Change Point Detectioncited by 2
  30. 2023SoftED: Metrics for Soft Evaluation of Time Series Event Detectioncited by 2
  31. 2024A Parameter-Efficient Quantum Anomaly Detection Method on a Superconducting Quantum Processorcited by 2
  32. 2025PatchTrAD: A Patch-Based Transformer focusing on Patch-Wise Reconstruction Error for Time Series Anomaly Detectioncited by 2
  33. 2021Novelty Detection in Sequential Data by Informed Clustering and Modelingcited by 1
  34. 2021Pastprop-RNN: improved predictions of the future by correcting the pastcited by 1
  35. 2023Impact of Deep Learning Libraries on Online Adaptive Lightweight Time Series Anomaly Detectioncited by 1
  36. 2023OrionBench: Benchmarking Time Series Generative Models in the Service of the End-Usercited by 1
  37. 2024CARE to Compare: A real-world dataset for anomaly detection in wind turbine datacited by 1
  38. 2024Exploring the Impact of Outlier Variability on Anomaly Detection Evaluation Metricscited by 1

50 most cited of 87 All 87 (JSON)