# Bridging the Gap: Unifying the Training and Evaluation of Neural Network Binary Classifiers

arXiv: https://arxiv.org/abs/2009.01367 · HTML: https://mlsanity.com/paper/2009.01367/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Sep 2020.

## Built on by (most cited first, 4 of 4)

- [Aligning Multiclass Neural Network Classifier Criterion with Task Performance Metrics](https://mlsanity.com/paper/2405.20954/) (2024)
- [Partial VOROS: A Cost-aware Performance Metric for Binary Classifiers with Precision and Capacity Constraints](https://mlsanity.com/paper/2510.18520/) (2025)
- [What Do Students Learn? A Feature-Level Analysis of Dark Knowledge](https://mlsanity.com/paper/2606.03052/) (2026)
- [SoftMCC: An MCC-Brier Calibration Bridge for Threshold-Free Model Selection under Class Imbalance](https://mlsanity.com/paper/2608.08984/) (2026)

## Abstract (arXiv)

While neural network binary classifiers are often evaluated on metrics such as Accuracy and F1-Score, they are commonly trained with a cross-entropy objective. How can this training-evaluation gap be addressed? While specific techniques have been adopted to optimize certain confusion matrix based metrics, it is challenging or impossible in some cases to generalize the techniques to other metrics. Adversarial learning approaches have also been proposed to optimize networks via confusion matrix based metrics, but they tend to be much slower than common training methods. In this work, we propose a unifying approach to training neural network binary classifiers that combines a differentiable approximation of the Heaviside function with a probabilistic view of the typical confusion matrix values using soft sets. Our theoretical analysis shows the benefit of using our method to optimize for a given evaluation metric, such as F1-Score, with soft sets, and our extensive experiments show the effectiveness of our approach in several domains.