# Normalized mutual information is a biased measure for classification and community detection

arXiv: https://arxiv.org/abs/2307.01282 · HTML: https://mlsanity.com/paper/2307.01282/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 69 references that are in the map.
- First posted on arXiv 3 Jul 2023.

## Builds on (most cited first, 15 of 15)

- [From Louvain to Leiden: guaranteeing well-connected communities](https://mlsanity.com/paper/1810.08473/) (2018)
- [Community detection in networks: A user guide](https://mlsanity.com/paper/1608.00163/) (2016)
- [Community detection thresholds and the weak Ramanujan property](https://mlsanity.com/paper/1311.3085/) (2013)
- [The ground truth about metadata and community detection in networks](https://mlsanity.com/paper/1608.05878/) (2016)
- [The Impact of Random Models on Clustering Similarity](https://mlsanity.com/paper/1701.06508/) (2017)
- [A Comparative Analysis of Community Detection Algorithms on Artificial Networks](https://mlsanity.com/paper/1608.00763/) (2016)
- [Nonparametric Bayesian inference of the microcanonical stochastic block model](https://mlsanity.com/paper/1610.02703/) (2016)
- [Prioritizing network communities](https://mlsanity.com/paper/1805.02411/) (2018)
- [Inference of hyperedges and overlapping communities in hypergraphs](https://mlsanity.com/paper/2204.05646/) (2022)
- [Evaluating accuracy of community detection using the relative normalized mutual information](https://mlsanity.com/paper/1501.03844/) (2015)
- [Revealing consensus and dissensus between network partitions](https://mlsanity.com/paper/2005.13977/) (2020)
- [Network community detection via neural embeddings](https://mlsanity.com/paper/2306.13400/) (2023)
- [Improved mutual information measure for classification and community detection](https://mlsanity.com/paper/1907.12581/) (2019)
- [Bayan Algorithm: Detecting Communities in Networks Through Exact and Approximate Optimization of Modularity](https://mlsanity.com/paper/2209.04562/) (2022)
- [A primer on information theory, with applications to neuroscience](https://mlsanity.com/paper/1304.2333/) (2013)

## Built on by (most cited first, 7 of 7)

- [Dial-In LLM: Human-Aligned LLM-in-the-loop Intent Clustering for Customer Service Dialogues](https://mlsanity.com/paper/2412.09049/) (2024)
- [Analyzing Modularity Maximization in Approximation, Heuristic, and Graph Neural Network Algorithms for Community Detection](https://mlsanity.com/paper/2310.10898/) (2023)
- [Matrix Factorization Framework for Community Detection under the Degree-Corrected Block Model](https://mlsanity.com/paper/2601.06262/) (2026)
- [Robust Markov stability for community detection at a scale learned based on the structure](https://mlsanity.com/paper/2504.11621/) (2025)
- [Assessing wildfire susceptibility in Iran: Leveraging machine learning for geospatial analysis of climatic and anthropogenic factors](https://mlsanity.com/paper/2505.14122/) (2025)
- [Unifying Information-Theoretic and Pair-Counting Clustering Similarity](https://mlsanity.com/paper/2511.03000/) (2025)
- [NMINE: Normalized Mutual Information Neural Estimation](https://mlsanity.com/paper/2607.27710/) (2026)

## Abstract (arXiv)

Normalized mutual information is widely used as a similarity measure for evaluating the performance of clustering and classification algorithms. In this paper, we argue that results returned by the normalized mutual information are biased for two reasons: first, because they ignore the information content of the contingency table and, second, because their symmetric normalization introduces spurious dependence on algorithm output. We introduce a modified version of the mutual information that remedies both of these shortcomings. As a practical demonstration of the importance of using an unbiased measure, we perform extensive numerical tests on a basket of popular algorithms for network community detection and show that one's conclusions about which algorithm is best are significantly affected by the biases in the traditional mutual information.