# Adjusted Similarity Measures and a Violation of Expectations

arXiv: https://arxiv.org/abs/2601.10641 · HTML: https://mlsanity.com/paper/2601.10641/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 28 references that are in the map.
- First posted on arXiv 15 Jan 2026.

## Builds on (most cited first, 6 of 6)

- [The Impact of Random Models on Clustering Similarity](https://mlsanity.com/paper/1701.06508/) (2017)
- [Adjusting for Chance Clustering Comparison Measures](https://mlsanity.com/paper/1512.01286/) (2015)
- [Characterizing and comparing external measures for the assessment of cluster analysis and community detection](https://mlsanity.com/paper/2102.00708/) (2021)
- [Multiresolution Consensus Clustering in Networks](https://mlsanity.com/paper/1710.02249/) (2017)
- [Random Models for Fuzzy Clustering Similarity Measures](https://mlsanity.com/paper/2312.10270/) (2023)
- [Estimating Agreement by Chance for Sequence Annotation](https://mlsanity.com/paper/2407.11371/) (2024)

## Abstract (arXiv)

Adjusted similarity measures, such as Cohen's kappa for inter-rater reliability and the adjusted Rand index used to compare clustering algorithms, are a vital tool for comparing discrete labellings. These measures are intended to have the property of 0 expectation under a null distribution and maximum value 1 under maximal similarity to aid in interpretation. Measures are frequently adjusted with respect to the permutation distribution for historic and analytic reasons. There is currently renewed interest in considering other null models more appropriate for context, such as clustering ensembles permitting a random number of identified clusters. The purpose of this work is two -- fold: (1) to generalize the study of the adjustment operator to general null models and to a more general procedure which includes statistical standardization as a special case and (2) to identify sufficient conditions for the adjustment operator to produce the intended properties, where sufficient conditions are related to whether and how observed data are incorporated into null distributions. We demonstrate how violations of the sufficient conditions may lead to substantial breakdown, such as by producing a non-positive measure under traditional adjustment rather than one with mean 0, or by producing a measure which is deterministically 0 under statistical standardization.