# A classification performance evaluation measure considering data separability

arXiv: https://arxiv.org/abs/2211.05433 · HTML: https://mlsanity.com/paper/2211.05433/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 10 Nov 2022.

## Builds on (most cited first, 8 of 8)

- [Learning Diverse and Discriminative Representations via the Principle of Maximal Coding Rate Reduction](https://mlsanity.com/paper/2006.08558/) (2020)
- [How Complex is your classification problem? A survey on measuring classification complexity](https://mlsanity.com/paper/1808.03591/) (2018)
- [Incremental Learning via Rate Reduction](https://mlsanity.com/paper/2011.14593/) (2020)
- [How deep is deep enough? -- Quantifying class separability in the hidden layers of deep neural networks](https://mlsanity.com/paper/1811.01753/) (2018)
- [A Rate-Distortion Framework for Explaining Neural Network Decisions](https://mlsanity.com/paper/1905.11092/) (2019)
- [Neural Complexity Measures](https://mlsanity.com/paper/2008.02953/) (2020)
- [A Novel Intrinsic Measure of Data Separability](https://mlsanity.com/paper/2109.05180/) (2021)
- [Reducing Data Complexity using Autoencoders with Class-informed Loss Functions](https://mlsanity.com/paper/2111.06142/) (2021)

## Built on by (most cited first, 1 of 1)

- [ChannelExplorer: Exploring Class Separability Through Activation Channel Visualization](https://mlsanity.com/paper/2505.04647/) (2025)

## Abstract (arXiv)

Machine learning and deep learning classification models are data-driven, and the model and the data jointly determine their classification performance. It is biased to evaluate the model's performance only based on the classifier accuracy while ignoring the data separability. Sometimes, the model exhibits excellent accuracy, which might be attributed to its testing on highly separable data. Most of the current studies on data separability measures are defined based on the distance between sample points, but this has been demonstrated to fail in several circumstances. In this paper, we propose a new separability measure--the rate of separability (RS), which is based on the data coding rate. We validate its effectiveness as a supplement to the separability measure by comparing it to four other distance-based measures on synthetic datasets. Then, we demonstrate the positive correlation between the proposed measure and recognition accuracy in a multi-task scenario constructed from a real dataset. Finally, we discuss the methods for evaluating the classification performance of machine learning and deep learning models considering data separability.