# A partition-based similarity for classification distributions

arXiv: https://arxiv.org/abs/2011.06557 · HTML: https://mlsanity.com/paper/2011.06557/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2020.

## Built on by (most cited first, 6 of 6)

- [Tracking the perspectives of interacting language models](https://mlsanity.com/paper/2406.11938/) (2024)
- [Statistical inference on black-box generative models in the data kernel perspective space](https://mlsanity.com/paper/2410.01106/) (2024)
- [A Statistical Turing Test for Generative Models](https://mlsanity.com/paper/2309.08913/) (2023)
- [Mental State Classification Using Multi-graph Features](https://mlsanity.com/paper/2203.00516/) (2022)
- [Inducing a hierarchy for multi-class classification problems](https://mlsanity.com/paper/2102.10263/) (2021)
- [Approximately optimal domain adaptation with Fisher's Linear Discriminant](https://mlsanity.com/paper/2302.14186/) (2023)

## Abstract (arXiv)

Herein we define a measure of similarity between classification distributions that is both principled from the perspective of statistical pattern recognition and useful from the perspective of machine learning practitioners. In particular, we propose a novel similarity on classification distributions, dubbed task similarity, that quantifies how an optimally-transformed optimal representation for a source distribution performs when applied to inference related to a target distribution. The definition of task similarity allows for natural definitions of adversarial and orthogonal distributions. We highlight limiting properties of representations induced by (universally) consistent decision rules and demonstrate in simulation that an empirical estimate of task similarity is a function of the decision rule deployed for inference. We demonstrate that for a given target distribution, both transfer efficiency and semantic similarity of candidate source distributions correlate with empirical task similarity.