# Objective evaluation metrics for automatic classification of EEG events

arXiv: https://arxiv.org/abs/1712.10107 · HTML: https://mlsanity.com/paper/1712.10107/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Dec 2017.

## Built on by (most cited first, 13 of 13)

- [SzCORE: A Seizure Community Open-source Research Evaluation framework for the validation of EEG-based automated seizure detection algorithms](https://mlsanity.com/paper/2402.13005/) (2024)
- [Systematic Assessment of Hyperdimensional Computing for Epileptic Seizure Detection](https://mlsanity.com/paper/2105.00934/) (2021)
- [Real-Time Seizure Detection using EEG: A Comprehensive Comparison of Recent Approaches under a Realistic Setting](https://mlsanity.com/paper/2201.08780/) (2022)
- [Multi-Centroid Hyperdimensional Computing Approach for Epileptic Seizure Detection](https://mlsanity.com/paper/2111.08463/) (2021)
- [Avoiding Post-Processing with Event-Based Detection in Biomedical Signals](https://mlsanity.com/paper/2209.11007/) (2022)
- [Exploration of Hyperdimensional Computing Strategies for Enhanced Learning on Epileptic Seizure Detection](https://mlsanity.com/paper/2201.09759/) (2022)
- [Hyperdimensional computing encoding for feature selection on the use case of epileptic seizure detection](https://mlsanity.com/paper/2205.07654/) (2022)
- [Low Latency Real-Time Seizure Detection Using Transfer Deep Learning](https://mlsanity.com/paper/2202.07796/) (2022)
- [Importance of methodological choices in data manipulation for validating epileptic seizure detection models](https://mlsanity.com/paper/2302.10672/) (2023)
- [Preictal Period Optimization for Deep Learning-Based Epileptic Seizure Prediction](https://mlsanity.com/paper/2407.14876/) (2024)
- [Adapting Tensor Kernel Machines to Enable Efficient Transfer Learning for Seizure Detection](https://mlsanity.com/paper/2512.02626/) (2025)
- [From Handcrafted Features to Functional Edge Learning: Evolution of EEG Seizure Detection Frameworks](https://mlsanity.com/paper/2606.22258/) (2026)
- [RobustSeiz: An Open-Source Framework for Benchmarking the Robustness of EEG Seizure Detection Models](https://mlsanity.com/paper/2609.04007/) (2026)

## Abstract (arXiv)

The evaluation of machine learning algorithms in biomedical fields for applications involving sequential data lacks standardization. Common quantitative scalar evaluation metrics such as sensitivity and specificity can often be misleading depending on the requirements of the application. Evaluation metrics must ultimately reflect the needs of users yet be sufficiently sensitive to guide algorithm development. Feedback from critical care clinicians who use automated event detection software in clinical applications has been overwhelmingly emphatic that a low false alarm rate, typically measured in units of the number of errors per 24 hours, is the single most important criterion for user acceptance. Though using a single metric is not often as insightful as examining performance over a range of operating conditions, there is a need for a single scalar figure of merit. In this paper, we discuss the deficiencies of existing metrics for a seizure detection task and propose several new metrics that offer a more balanced view of performance. We demonstrate these metrics on a seizure detection task based on the TUH EEG Corpus. We show that two promising metrics are a measure based on a concept borrowed from the spoken term detection literature, Actual Term-Weighted Value (ATWV), and a new metric, Time-Aligned Event Scoring (TAES), that accounts for the temporal alignment of the hypothesis to the reference annotation. We also demonstrate that state of the art technology based on deep learning, though impressive in its performance, still needs significant improvement before it meets very strict user acceptance criteria.