# Selective Inference for Hierarchical Clustering

arXiv: https://arxiv.org/abs/2012.02936 · HTML: https://mlsanity.com/paper/2012.02936/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2020.

## Built on by (most cited first, 30 of 39)

- [Selective inference for k-means clustering](https://mlsanity.com/paper/2203.15267/) (2022)
- [Tree-Values: selective inference for regression trees](https://mlsanity.com/paper/2106.07816/) (2021)
- [Data thinning for convolution-closed distributions](https://mlsanity.com/paper/2301.07276/) (2023)
- [CAD-DA: Controllable Anomaly Detection after Domain Adaptation by Statistical Inference](https://mlsanity.com/paper/2310.14608/) (2023)
- [Generalized Data Thinning Using Sufficient Statistics](https://mlsanity.com/paper/2303.12931/) (2023)
- [Interpretable Machine Learning for Discovery: Statistical Challenges & Opportunities](https://mlsanity.com/paper/2308.01475/) (2023)
- [Statistical Test for Anomaly Detections by Variational Auto-Encoders](https://mlsanity.com/paper/2402.03724/) (2024)
- [Exact Selective Inference with Randomization](https://mlsanity.com/paper/2212.12940/) (2022)
- [Statistical Test for Attention Map in Vision Transformer](https://mlsanity.com/paper/2401.08169/) (2024)
- [Kernel Biclustering algorithm in Hilbert Spaces](https://mlsanity.com/paper/2208.03675/) (2022)
- [Statistical Test for Feature Selection Pipelines by Selective Inference](https://mlsanity.com/paper/2406.18902/) (2024)
- [Decomposing Gaussians with Unknown Covariance](https://mlsanity.com/paper/2409.11497/) (2024)
- [Statistical Test for Auto Feature Engineering by Selective Inference](https://mlsanity.com/paper/2410.19768/) (2024)
- [Quantifying Statistical Significance of Deep Nearest Neighbor Anomaly Detection via Selective Inference](https://mlsanity.com/paper/2502.12978/) (2025)
- [Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices](https://mlsanity.com/paper/2506.04553/) (2025)
- [Controllable RANSAC-based Anomaly Detection via Hypothesis Testing](https://mlsanity.com/paper/2410.15133/) (2024)
- [Change Point Detection in the Frequency Domain with Statistical Reliability](https://mlsanity.com/paper/2502.03062/) (2025)
- [Statistical Inference for Autoencoder-based Anomaly Detection after Representation Learning-based Domain Adaptation](https://mlsanity.com/paper/2508.07049/) (2025)
- [Weighted Conformal Clustering](https://mlsanity.com/paper/2606.00436/) (2026)
- [Controlling the False Split Rate in Tree-Based Aggregation](https://mlsanity.com/paper/2108.05350/) (2021)
- [When Does Bottom-up Beat Top-down in Hierarchical Community Detection?](https://mlsanity.com/paper/2306.00833/) (2023)
- [Bounded P-values in Parametric Programming-based Selective Inference](https://mlsanity.com/paper/2307.11351/) (2023)
- [Statistical Test for Diffusion-Based Anomaly Localization via Selective Inference](https://mlsanity.com/paper/2402.11789/) (2024)
- [Forest Fire Clustering for Single-cell Sequencing with Iterative Label Propagation and Parallelized Monte Carlo Simulation](https://mlsanity.com/paper/2103.11802/) (2021)
- [Statistical Inference for the Dynamic Time Warping Distance, with Application to Abnormal Time-Series Detection](https://mlsanity.com/paper/2202.06593/) (2022)
- [Inferring independent sets of Gaussian variables after thresholding correlations](https://mlsanity.com/paper/2211.01521/) (2022)
- [Interpretable Clustering with the Distinguishability Criterion](https://mlsanity.com/paper/2404.15967/) (2024)
- [Statistical Inference for Clustering-based Anomaly Detection](https://mlsanity.com/paper/2504.18633/) (2025)
- [Flexible Selective Inference with Flow-based Transport Maps](https://mlsanity.com/paper/2506.01150/) (2025)
- [Two-cluster test](https://mlsanity.com/paper/2507.08382/) (2025)

## Abstract (arXiv)

Classical tests for a difference in means control the type I error rate when the groups are defined a priori. However, when the groups are instead defined via clustering, then applying a classical test yields an extremely inflated type I error rate. Notably, this problem persists even if two separate and independent data sets are used to define the groups and to test for a difference in their means. To address this problem, in this paper, we propose a selective inference approach to test for a difference in means between two clusters. Our procedure controls the selective type I error rate by accounting for the fact that the choice of null hypothesis was made based on the data. We describe how to efficiently compute exact p-values for clusters obtained using agglomerative hierarchical clustering with many commonly-used linkages. We apply our method to simulated data and to single-cell RNA-sequencing data.