# SoftED: Metrics for Soft Evaluation of Time Series Event Detection

arXiv: https://arxiv.org/abs/2304.00439 · HTML: https://mlsanity.com/paper/2304.00439/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 84 references that are in the map.
- First posted on arXiv 2 Apr 2023.

## Builds on (most cited first, 12 of 12)

- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [A review on outlier/anomaly detection in time series data](https://mlsanity.com/paper/2002.04236/) (2020)
- [Deep Learning for Anomaly Detection: A Review](https://mlsanity.com/paper/2007.02500/) (2020)
- [Evaluating Real-time Anomaly Detection Algorithms - the Numenta Anomaly Benchmark](https://mlsanity.com/paper/1510.03336/) (2015)
- [Local Evaluation of Time Series Anomaly Detection Algorithms](https://mlsanity.com/paper/2206.13167/) (2022)
- [Precision and Recall for Time Series](https://mlsanity.com/paper/1803.03639/) (2018)
- [Anomaly Detection in Univariate Time-series: A Survey on the State-of-the-Art](https://mlsanity.com/paper/2004.00433/) (2020)
- [Exathlon: A Benchmark for Explainable Anomaly Detection over Time Series](https://mlsanity.com/paper/2010.05073/) (2020)
- [Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series](https://mlsanity.com/paper/2303.01272/) (2023)
- [Series2Graph: Graph-based Subsequence Anomaly Detection for Time Series](https://mlsanity.com/paper/2207.12208/) (2022)
- [Event Prediction in the Big Data Era: A Systematic Survey](https://mlsanity.com/paper/2007.09815/) (2020)
- [On the Runtime-Efficacy Trade-off of Anomaly Detection Techniques for Real-Time Streaming Data](https://mlsanity.com/paper/1710.04735/) (2017)

## Built on by (most cited first, 2 of 2)

- [Using deep neural networks to detect non-analytically defined expert event labels in canoe sprint force sensor signals](https://mlsanity.com/paper/2407.08395/) (2024)
- [Accurate Evaluation of Quickest Changepoint Detectors via Non-parametric Survival Analysis](https://mlsanity.com/paper/2605.18798/) (2026)

## Abstract (arXiv)

Time series event detection methods are evaluated mainly by standard classification metrics that focus solely on detection accuracy. However, inaccuracy in detecting an event can often result from its preceding or delayed effects reflected in neighboring detections. These detections are valuable to trigger necessary actions or help mitigate unwelcome consequences. In this context, current metrics are insufficient and inadequate for the context of event detection. There is a demand for metrics that incorporate both the concept of time and temporal tolerance for neighboring detections. This paper introduces SoftED metrics, a new set of metrics designed for soft evaluating event detection methods. They enable the evaluation of both detection accuracy and the degree to which their detections represent events. They improved event detection evaluation by associating events and their representative detections, incorporating temporal tolerance in over 36% of experiments compared to the usual classification metrics. SoftED metrics were validated by domain specialists that indicated their contribution to detection evaluation and method selection.