# Large Deviations for Classification Performance Analysis of Machine Learning Systems

arXiv: https://arxiv.org/abs/2301.07104 · HTML: https://mlsanity.com/paper/2301.07104/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 16 Jan 2023.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Learning to Detect](https://mlsanity.com/paper/1805.07631/) (2018)
- [Deep Learning Methods for Vessel Trajectory Prediction based on Recurrent Neural Networks](https://mlsanity.com/paper/2101.02486/) (2021)
- [Active Sensing for Communications by Learning](https://mlsanity.com/paper/2112.04075/) (2021)
- [Space-based Global Maritime Surveillance. Part I: Satellite Technologies](https://mlsanity.com/paper/2011.11304/) (2020)
- [Space-based Global Maritime Surveillance. Part II: Artificial Intelligence and Data Fusion Techniques](https://mlsanity.com/paper/2011.11338/) (2020)
- [Statistical Hypothesis Testing Based on Machine Learning: Large Deviations Analysis](https://mlsanity.com/paper/2207.10939/) (2022)

## Abstract (arXiv)

We study the performance of machine learning binary classification techniques in terms of error probabilities. The statistical test is based on the Data-Driven Decision Function (D3F), learned in the training phase, i.e., what is thresholded before the final binary decision is made. Based on large deviations theory, we show that under appropriate conditions the classification error probabilities vanish exponentially, as ~ expleft(-n,I + o(n) right), where I is the error rate and n is the number of observations available for testing. We also propose two different approximations for the error probability curves, one based on a refined asymptotic formula (often referred to as exact asymptotics), and another one based on the central limit theorem. The theoretical findings are finally tested using the popular MNIST dataset.