mlsanityData to 14 Oct 2026

On the Hardness of Conditional Independence Testing In Practice

arXiv 2512.14000 ↗First posted 16 Dec 2025stat.ML cs.LG stat.MEcited by 7

Not on the map. Why?

Abstract (arXiv)

Tests of conditional independence (CI) underpin a number of important problems in machine learning and statistics, from causal discovery to evaluation of predictor fairness and out-of-distribution robustness. Shah and Peters (2020) showed that, contrary to the unconditional case, no universally finite-sample valid test can ever achieve nontrivial power. While informative, this result (based on "hiding" dependence) does not seem to explain the frequent practical failures observed with popular CI tests. We investigate the Kernel-based Conditional Independence (KCI) test - of which we show the Generalized Covariance Measure underlying many recent tests is nearly a special case - and identify the major factors underlying its practical behavior. We highlight the key role of errors in the conditional mean embedding estimate for the Type-I error, while pointing out the importance of selecting an appropriate conditioning kernel (not recognized in previous work) as being necessary for good test power but also tending to inflate Type-I error.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 21 of its 52 references that are in the map. First posted on arXiv 16 Dec 2025.

Builds on

21 of 52 references.

References · 21

  1. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  2. 2017Age Progression/Regression by Conditional Adversarial Autoencodercited by 429
  3. 2020Learning Deep Kernels for Non-Parametric Two-Sample Testscited by 130
  4. 2016Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selectioncited by 121
  5. 2016Generative Models and Model Criticism via Optimized Maximum Mean Discrepancycited by 100
  6. 2020A Measure-Theoretic Approach to Kernel Conditional Mean Embeddingscited by 86
  7. 2016Interpretable Distribution Features with Maximum Testing Powercited by 63
  8. 2021Proximal Causal Learning with Kernels: Two-Stage Estimation and Moment Restrictioncited by 55
  9. 2017Model-Powered Conditional Independence Testcited by 43
  10. 2017Characteristic and Universal Tensor Product Kernelscited by 41
  11. 2019A Rigorous Theory of Conditional Mean Embeddingscited by 31
  12. 2021Local permutation tests for conditional independencecited by 23
9 more
  1. 2023Towards Optimal Sobolev Norm Rates for the Vector-Valued Regularized Least-Squares Algorithmcited by 22
  2. 2022Efficient Conditionally Invariant Representation Learningcited by 19
  3. 2024Practical Kernel Tests of Conditional Independencecited by 15
  4. 2021The Weighted Generalised Covariance Measurecited by 13
  5. 2022MMD-B-Fair: Learning Fair Representations with Statistical Testingcited by 13
  6. 2021An Asymptotic Test for Conditional Independence using Analytic Kernel Embeddingscited by 12
  7. 2022Anytime Valid Tests of Conditional Independence Under Model-Xcited by 11
  8. 2023Conditional independence testing under misspecified inductive biasescited by 7
  9. 2025Testing Conditional Mean Independence Using Generative Neural Networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2026The Generalised Kernel Covariance Measurecited by 2
  2. 2025Toward Scalable and Valid Conditional Independence Testing with Spectral Representationscited by 1
  3. 2026Robust Weighted Triangulation of Causal Effects Under Model Uncertaintycited by 1
  4. 2026Fast Flow Matching based Conditional Independence Tests for Causal Discoverycited by 0
  5. 2026Masking Causality and Conditional Dependencecited by 0
  6. 2026Semiparametrically Efficient Inference for Kernel Measures of Noise Heterogeneitycited by 0
  7. 2026Sequential Kernel-based Conditional Independence Testing via Adaptive Bettingcited by 0