# On the Hardness of Conditional Independence Testing In Practice

arXiv: https://arxiv.org/abs/2512.14000 · HTML: https://mlsanity.com/paper/2512.14000/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 52 references that are in the map.
- First posted on arXiv 16 Dec 2025.

## Builds on (most cited first, 21 of 21)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Learning Deep Kernels for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2002.09116/) (2020)
- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy](https://mlsanity.com/paper/1611.04488/) (2016)
- [A Measure-Theoretic Approach to Kernel Conditional Mean Embeddings](https://mlsanity.com/paper/2002.03689/) (2020)
- [Interpretable Distribution Features with Maximum Testing Power](https://mlsanity.com/paper/1605.06796/) (2016)
- [Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restriction](https://mlsanity.com/paper/2105.04544/) (2021)
- [Model-Powered Conditional Independence Test](https://mlsanity.com/paper/1709.06138/) (2017)
- [Characteristic and Universal Tensor Product Kernels](https://mlsanity.com/paper/1708.08157/) (2017)
- [A Rigorous Theory of Conditional Mean Embeddings](https://mlsanity.com/paper/1912.00671/) (2019)
- [Local permutation tests for conditional independence](https://mlsanity.com/paper/2112.11666/) (2021)
- [Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithm](https://mlsanity.com/paper/2312.07186/) (2023)
- [Efficient Conditionally Invariant Representation Learning](https://mlsanity.com/paper/2212.08645/) (2022)
- [Practical Kernel Tests of Conditional Independence](https://mlsanity.com/paper/2402.13196/) (2024)
- [The Weighted Generalised Covariance Measure](https://mlsanity.com/paper/2111.04361/) (2021)
- [MMD-B-Fair: Learning Fair Representations with Statistical Testing](https://mlsanity.com/paper/2211.07907/) (2022)
- [An Asymptotic Test for Conditional Independence using Analytic Kernel Embeddings](https://mlsanity.com/paper/2110.14868/) (2021)
- [Anytime Valid Tests of Conditional Independence Under Model-X](https://mlsanity.com/paper/2209.12637/) (2022)
- [Conditional independence testing under misspecified inductive biases](https://mlsanity.com/paper/2307.02520/) (2023)
- [Testing Conditional Mean Independence Using Generative Neural Networks](https://mlsanity.com/paper/2501.17345/) (2025)

## Built on by (most cited first, 7 of 7)

- [The Generalised Kernel Covariance Measure](https://mlsanity.com/paper/2604.03721/) (2026)
- [Toward Scalable and Valid Conditional Independence Testing with Spectral Representations](https://mlsanity.com/paper/2512.19510/) (2025)
- [Robust Weighted Triangulation of Causal Effects Under Model Uncertainty](https://mlsanity.com/paper/2603.01119/) (2026)
- [Fast Flow Matching based Conditional Independence Tests for Causal Discovery](https://mlsanity.com/paper/2602.08315/) (2026)
- [Masking Causality and Conditional Dependence](https://mlsanity.com/paper/2603.06984/) (2026)
- [Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneity](https://mlsanity.com/paper/2605.27526/) (2026)
- [Sequential Kernel-based Conditional Independence Testing via Adaptive Betting](https://mlsanity.com/paper/2606.18993/) (2026)

## Abstract (arXiv)

Tests of conditional independence (CI) underpin a number of important problems in machine learning and statistics, from causal discovery to evaluation of predictor fairness and out-of-distribution robustness. Shah and Peters (2020) showed that, contrary to the unconditional case, no universally finite-sample valid test can ever achieve nontrivial power. While informative, this result (based on "hiding" dependence) does not seem to explain the frequent practical failures observed with popular CI tests. We investigate the Kernel-based Conditional Independence (KCI) test - of which we show the Generalized Covariance Measure underlying many recent tests is nearly a special case - and identify the major factors underlying its practical behavior. We highlight the key role of errors in the conditional mean embedding estimate for the Type-I error, while pointing out the importance of selecting an appropriate conditioning kernel (not recognized in previous work) as being necessary for good test power but also tending to inflate Type-I error.