# Towards Unbiased Evaluation of Time-series Anomaly Detector

arXiv: https://arxiv.org/abs/2409.13053 · HTML: https://mlsanity.com/paper/2409.13053/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 19 Sep 2024.

## Builds on (most cited first, 15 of 15)

- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [MOMENT: A Family of Open Time-series Foundation Models](https://mlsanity.com/paper/2402.03885/) (2024)
- [Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy](https://mlsanity.com/paper/2110.02642/) (2021)
- [Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications](https://mlsanity.com/paper/1802.03903/) (2018)
- [Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress](https://mlsanity.com/paper/2009.13807/) (2020)
- [Deep Learning for Time Series Anomaly Detection: A Survey](https://mlsanity.com/paper/2211.05244/) (2022)
- [Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT](https://mlsanity.com/paper/2104.03466/) (2021)
- [Towards a Rigorous Evaluation of Time-series Anomaly Detection](https://mlsanity.com/paper/2109.05257/) (2021)
- [An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series](https://mlsanity.com/paper/2109.11428/) (2021)
- [Precision and Recall for Time Series](https://mlsanity.com/paper/1803.03639/) (2018)
- [TFAD: A Decomposition Time Series Anomaly Detection Architecture with Time-Frequency Analysis](https://mlsanity.com/paper/2210.09693/) (2022)
- [Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series](https://mlsanity.com/paper/2303.01272/) (2023)
- [CARLA: Self-supervised Contrastive Representation Learning for Time Series Anomaly Detection](https://mlsanity.com/paper/2308.09296/) (2023)
- [Robust Learning of Deep Time Series Anomaly Detection Models with Contaminated Training Data](https://mlsanity.com/paper/2208.01841/) (2022)
- [Self-Supervised Time-Series Anomaly Detection Using Learnable Data Augmentation](https://mlsanity.com/paper/2406.12260/) (2024)

## Built on by (most cited first, 2 of 2)

- [PaAno: Patch-Based Representation Learning for Time-Series Anomaly Detection](https://mlsanity.com/paper/2602.01359/) (2026)
- [HarmoniAD: Harmonizing Local Structures and Global Semantics for Anomaly Detection](https://mlsanity.com/paper/2601.00327/) (2026)

## Abstract (arXiv)

Time series anomaly detection (TSAD) is an evolving area of research motivated by its critical applications, such as detecting seismic activity, sensor failures in industrial plants, predicting crashes in the stock market, and so on. Across domains, anomalies occur significantly less frequently than normal data, making the F1-score the most commonly adopted metric for anomaly detection. However, in the case of time series, it is not straightforward to use standard F1-score because of the dissociation between `time points' and `time events'. To accommodate this, anomaly predictions are adjusted, called as point adjustment (PA), before the F1-score evaluation. However, these adjustments are heuristics-based, and biased towards true positive detection, resulting in over-estimated detector performance. In this work, we propose an alternative adjustment protocol called ``Balanced point adjustment'' (BA). It addresses the limitations of existing point adjustment methods and provides guarantees of fairness backed by axiomatic definitions of TSAD evaluation.