Selective Inference for Hierarchical Clustering
Not on the map. Why?
Abstract (arXiv)
Classical tests for a difference in means control the type I error rate when the groups are defined a priori. However, when the groups are instead defined via clustering, then applying a classical test yields an extremely inflated type I error rate. Notably, this problem persists even if two separate and independent data sets are used to define the groups and to test for a difference in their means. To address this problem, in this paper, we propose a selective inference approach to test for a difference in means between two clusters. Our procedure controls the selective type I error rate by accounting for the fact that the choice of null hypothesis was made based on the data. We describe how to efficiently compute exact p-values for clusters obtained using agglomerative hierarchical clustering with many commonly-used linkages. We apply our method to simulated data and to single-cell RNA-sequencing data.
Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Dec 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 5; 2023: 6; 2024: 8; 2025: 8; 2026: 7.
37 papers.
Sort:
- 2022Selective inference for k-means clusteringcited by 21
- 2021Tree-Values: selective inference for regression treescited by 15
- 2023Data thinning for convolution-closed distributionscited by 12
- 2023CAD-DA: Controllable Anomaly Detection after Domain Adaptation by Statistical Inferencecited by 10
- 2023Generalized Data Thinning Using Sufficient Statisticscited by 7
- 2023Interpretable Machine Learning for Discovery: Statistical Challenges & Opportunitiescited by 7
- 2022Exact Selective Inference with Randomizationcited by 6
- 2024Statistical Test for Attention Map in Vision Transformercited by 6
- 2024Statistical Test for Anomaly Detections by Variational Auto-Encoderscited by 6
- 2022Kernel Biclustering algorithm in Hilbert Spacescited by 3
- 2024Statistical Test for Feature Selection Pipelines by Selective Inferencecited by 3
- 2024Statistical Test for Auto Feature Engineering by Selective Inferencecited by 3
25 more
- 2025Quantifying Statistical Significance of Deep Nearest Neighbor Anomaly Detection via Selective Inferencecited by 3
- 2024Decomposing Gaussians with Unknown Covariancecited by 2
- 2024Controllable RANSAC-based Anomaly Detection via Hypothesis Testingcited by 2
- 2025Change Point Detection in the Frequency Domain with Statistical Reliabilitycited by 2
- 2025Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practicescited by 2
- 2025Statistical Inference for Autoencoder-based Anomaly Detection after Representation Learning-based Domain Adaptationcited by 2
- 2021Controlling the False Split Rate in Tree-Based Aggregationcited by 1
- 2023When Does Bottom-up Beat Top-down in Hierarchical Community Detection?cited by 1
- 2023Bounded P-values in Parametric Programming-based Selective Inferencecited by 1
- 2024Statistical Test for Diffusion-Based Anomaly Localization via Selective Inferencecited by 1
- 2026Weighted Conformal Clusteringcited by 1
- 2021Forest Fire Clustering for Single-cell Sequencing with Iterative Label Propagation and Parallelized Monte Carlo Simulationcited by 0
- 2022Statistical Inference for the Dynamic Time Warping Distance, with Application to Abnormal Time-Series Detectioncited by 0
- 2022Inferring independent sets of Gaussian variables after thresholding correlationscited by 0
- 2024Interpretable Clustering with the Distinguishability Criterioncited by 0
- 2025Statistical Inference for Clustering-based Anomaly Detectioncited by 0
- 2025Flexible Selective Inference with Flow-based Transport Mapscited by 0
- 2025Two-cluster testcited by 0
- 2025Hierarchical Clustering With Confidencecited by 0
- 2026The Condition-Number Principle for Prototype Clusteringcited by 0
- 2026Testing hypotheses via orthogonalizationcited by 0
- 2026Automatic Statistical Test for Rationally Expressible Algorithms by Selective Inference, with Applications to Feature Selectioncited by 0
- 2026Nonparametric Hypothesis Testing of High-dimensional Clustering With Application to Single-cell RNA Datacited by 0
- 2026Statistical Testing for Multiple Instance Learning via Selective Inference with Applications to Computational Pathologycited by 0
- 2026Post-Anomaly Detection Inference for Deep SVDDcited by 0
- 2026Post-Anomaly Detection Inference for Deep SVDD29 Sep 2026
- 2026Statistical Testing for Multiple Instance Learning via Selective Inference with Applications to Computational Pathology21 Sep 2026
- 2026Nonparametric Hypothesis Testing of High-dimensional Clustering With Application to Single-cell RNA Data4 Sep 2026
- 2026Automatic Statistical Test for Rationally Expressible Algorithms by Selective Inference, with Applications to Feature Selection5 Aug 2026
- 2026Testing hypotheses via orthogonalization29 Jun 2026
- 2026Weighted Conformal Clustering29 May 2026
- 2026The Condition-Number Principle for Prototype Clustering9 Apr 2026
- 2025Hierarchical Clustering With Confidence6 Dec 2025
- 2025Statistical Inference for Autoencoder-based Anomaly Detection after Representation Learning-based Domain Adaptation9 Aug 2025
- 2025Two-cluster test11 Jul 2025
- 2025Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices5 Jun 2025
- 2025Flexible Selective Inference with Flow-based Transport Maps1 Jun 2025
25 more
- 2025Statistical Inference for Clustering-based Anomaly Detection25 Apr 2025
- 2025Quantifying Statistical Significance of Deep Nearest Neighbor Anomaly Detection via Selective Inference18 Feb 2025
- 2025Change Point Detection in the Frequency Domain with Statistical Reliability5 Feb 2025
- 2024Controllable RANSAC-based Anomaly Detection via Hypothesis Testing19 Oct 2024
- 2024Statistical Test for Auto Feature Engineering by Selective Inference13 Oct 2024
- 2024Decomposing Gaussians with Unknown Covariance17 Sep 2024
- 2024Statistical Test for Feature Selection Pipelines by Selective Inference27 Jun 2024
- 2024Interpretable Clustering with the Distinguishability Criterion24 Apr 2024
- 2024Statistical Test for Diffusion-Based Anomaly Localization via Selective Inference19 Feb 2024
- 2024Statistical Test for Anomaly Detections by Variational Auto-Encoders6 Feb 2024
- 2024Statistical Test for Attention Map in Vision Transformer16 Jan 2024
- 2023CAD-DA: Controllable Anomaly Detection after Domain Adaptation by Statistical Inference23 Oct 2023
- 2023Interpretable Machine Learning for Discovery: Statistical Challenges & Opportunities2 Aug 2023
- 2023Bounded P-values in Parametric Programming-based Selective Inference21 Jul 2023
- 2023When Does Bottom-up Beat Top-down in Hierarchical Community Detection?1 Jun 2023
- 2023Generalized Data Thinning Using Sufficient Statistics22 Mar 2023
- 2023Data thinning for convolution-closed distributions18 Jan 2023
- 2022Exact Selective Inference with Randomization25 Dec 2022
- 2022Inferring independent sets of Gaussian variables after thresholding correlations2 Nov 2022
- 2022Kernel Biclustering algorithm in Hilbert Spaces7 Aug 2022
- 2022Selective inference for k-means clustering29 Mar 2022
- 2022Statistical Inference for the Dynamic Time Warping Distance, with Application to Abnormal Time-Series Detection14 Feb 2022
- 2021Controlling the False Split Rate in Tree-Based Aggregation11 Aug 2021
- 2021Tree-Values: selective inference for regression trees15 Jun 2021
- 2021Forest Fire Clustering for Single-cell Sequencing with Iterative Label Propagation and Parallelized Monte Carlo Simulation22 Mar 2021
- 2022Selective inference for k-means clusteringcited by 21
- 2021Tree-Values: selective inference for regression treescited by 15
- 2023Data thinning for convolution-closed distributionscited by 12
- 2023CAD-DA: Controllable Anomaly Detection after Domain Adaptation by Statistical Inferencecited by 10
- 2023Generalized Data Thinning Using Sufficient Statisticscited by 7
- 2023Interpretable Machine Learning for Discovery: Statistical Challenges & Opportunitiescited by 7
- 2022Exact Selective Inference with Randomizationcited by 6
- 2024Statistical Test for Attention Map in Vision Transformercited by 6
- 2024Statistical Test for Anomaly Detections by Variational Auto-Encoderscited by 6
- 2022Kernel Biclustering algorithm in Hilbert Spacescited by 3
- 2024Statistical Test for Feature Selection Pipelines by Selective Inferencecited by 3
- 2024Statistical Test for Auto Feature Engineering by Selective Inferencecited by 3
12 most cited of 37 All 37 (JSON)
- 2026Post-Anomaly Detection Inference for Deep SVDD29 Sep 2026
- 2026Statistical Testing for Multiple Instance Learning via Selective Inference with Applications to Computational Pathology21 Sep 2026
- 2026Nonparametric Hypothesis Testing of High-dimensional Clustering With Application to Single-cell RNA Data4 Sep 2026
- 2026Automatic Statistical Test for Rationally Expressible Algorithms by Selective Inference, with Applications to Feature Selection5 Aug 2026
- 2026Testing hypotheses via orthogonalization29 Jun 2026
- 2026Weighted Conformal Clustering29 May 2026
- 2026The Condition-Number Principle for Prototype Clustering9 Apr 2026
- 2025Hierarchical Clustering With Confidence6 Dec 2025
- 2025Statistical Inference for Autoencoder-based Anomaly Detection after Representation Learning-based Domain Adaptation9 Aug 2025
- 2025Two-cluster test11 Jul 2025
- 2025Unsupervised Machine Learning for Scientific Discovery: Workflow and Best Practices5 Jun 2025
- 2025Flexible Selective Inference with Flow-based Transport Maps1 Jun 2025
12 newest of 37 All 37 (JSON)