# Measure of Strength of Evidence for Visually Observed Differences between Subpopulations

arXiv: https://arxiv.org/abs/2101.00362 · HTML: https://mlsanity.com/paper/2101.00362/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 20 references that are in the map.
- First posted on arXiv 2 Jan 2021.

## Builds on (most cited first, 2 of 2)

- [Direction-Projection-Permutation for High Dimensional Hypothesis Tests](https://mlsanity.com/paper/1304.0796/) (2013)
- [Fast algorithms for large scale generalized distance weighted discrimination](https://mlsanity.com/paper/1604.05473/) (2016)

## Abstract (arXiv)

For measuring the strength of visually-observed subpopulation differences, the Population Difference Criterion is proposed to assess the statistical significance of visually observed subpopulation differences. It addresses the following challenges: in high-dimensional contexts, distributional models can be dubious; in high-signal contexts, conventional permutation tests give poor pairwise comparisons. We also make two other contributions: Based on a careful analysis we find that a balanced permutation approach is more powerful in high-signal contexts than conventional permutations. Another contribution is the quantification of uncertainty due to permutation variation via a bootstrap confidence interval. The practical usefulness of these ideas is illustrated in the comparison of subpopulations of modern cancer data.