# On the Normalization of Confusion Matrices: Methods and Geometric Interpretations

arXiv: https://arxiv.org/abs/2509.04959 · HTML: https://mlsanity.com/paper/2509.04959/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 51 references that are in the map.
- First posted on arXiv 5 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity](https://mlsanity.com/paper/2302.01772/) (2023)
- [Synthetic Examples Improve Generalization for Rare Classes](https://mlsanity.com/paper/1904.05916/) (2019)
- [Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels](https://mlsanity.com/paper/2110.12536/) (2021)
- [Minority Class Oriented Active Learning for Imbalanced Datasets](https://mlsanity.com/paper/2202.00390/) (2022)
- [Solving the Class Imbalance Problem Using a Counterfactual Method for Data Augmentation](https://mlsanity.com/paper/2111.03516/) (2021)
- [Deep Span Representations for Named Entity Recognition](https://mlsanity.com/paper/2210.04182/) (2022)

## Abstract (arXiv)

The confusion matrix is a standard tool for evaluating classifiers by providing insights into class-level errors. In heterogeneous settings, its values are shaped by two main factors: class similarity -- how easily the model confuses two classes -- and distribution bias, arising from skewed distributions in the training and test sets. However, confusion matrix values reflect a mix of both factors, making it difficult to disentangle their individual contributions. To address this, we introduce bistochastic normalization using Iterative Proportional Fitting, a generalization of row and column normalization. Unlike standard normalizations, this method recovers the underlying structure of class similarity. By disentangling error sources, it enables more accurate diagnosis of model behavior and supports more targeted improvements. We also show a correspondence between confusion matrix normalizations and the model's internal class representations. Both standard and bistochastic normalizations can be interpreted geometrically in this space, offering a deeper understanding of what normalization reveals about a classifier.