# A Conditional Randomization Test for Sparse Logistic Regression in High-Dimension

arXiv: https://arxiv.org/abs/2205.14613 · HTML: https://mlsanity.com/paper/2205.14613/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 42 references that are in the map.
- First posted on arXiv 29 May 2022.

## Builds on (most cited first, 10 of 10)

- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [A modern maximum-likelihood theory for high-dimensional logistic regression](https://mlsanity.com/paper/1803.06964/) (2018)
- [Confidence Intervals and Hypothesis Testing for High-Dimensional Regression](https://mlsanity.com/paper/1306.3171/) (2013)
- [Finite-sample analysis of M-estimators using self-concordance](https://mlsanity.com/paper/1810.06838/) (2018)
- [A General Theory of Hypothesis Tests and Confidence Regions for Sparse High Dimensional Models](https://mlsanity.com/paper/1412.8765/) (2014)
- [Fast and Powerful Conditional Randomization Testing via Distillation](https://mlsanity.com/paper/2006.03980/) (2020)
- [The Asymptotic Distribution of the MLE in High-dimensional Logistic Models: Arbitrary Covariance](https://mlsanity.com/paper/2001.09351/) (2020)
- [SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression](https://mlsanity.com/paper/2103.12725/) (2021)
- [Spatially relaxed inference on high-dimensional linear models](https://mlsanity.com/paper/2106.02590/) (2021)

## Built on by (most cited first, 2 of 2)

- [Statistically Valid Variable Importance Assessment through Conditional Permutations](https://mlsanity.com/paper/2309.07593/) (2023)
- [Measuring Variable Importance in Heterogeneous Treatment Effects with Confidence](https://mlsanity.com/paper/2408.13002/) (2024)

## Abstract (arXiv)

Identifying the relevant variables for a classification model with correct confidence levels is a central but difficult task in high-dimension. Despite the core role of sparse logistic regression in statistics and machine learning, it still lacks a good solution for accurate inference in the regime where the number of features p is as large as or larger than the number of samples n. Here, we tackle this problem by improving the Conditional Randomization Test (CRT). The original CRT algorithm shows promise as a way to output p-values while making few assumptions on the distribution of the test statistics. As it comes with a prohibitive computational cost even in mildly high-dimensional problems, faster solutions based on distillation have been proposed. Yet, they rely on unrealistic hypotheses and result in low-power solutions. To improve this, we propose CRT-logit, an algorithm that combines a variable-distillation step and a decorrelation step that takes into account the geometry of ℓ1-penalized logistic regression problem. We provide a theoretical analysis of this procedure, and demonstrate its effectiveness on simulations, along with experiments on large-scale brain-imaging and genomics datasets.