# Reinterpreting causal discovery as the task of predicting unobserved joint statistics

arXiv: https://arxiv.org/abs/2305.06894 · HTML: https://mlsanity.com/paper/2305.06894/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 65 references that are in the map.
- First posted on arXiv 11 May 2023.

## Builds on (most cited first, 17 of 17)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Just Interpolate: Kernel "Ridgeless" Regression Can Generalize](https://mlsanity.com/paper/1808.00387/) (2018)
- [Causal Inference and Causal Explanation with Background Knowledge](https://mlsanity.com/paper/1302.4972/) (2013)
- [Benign overfitting in ridge regression](https://mlsanity.com/paper/2009.14286/) (2020)
- [Harmless interpolation of noisy data in regression](https://mlsanity.com/paper/1903.09139/) (2019)
- [Does data interpolation contradict statistical optimality?](https://mlsanity.com/paper/1806.09471/) (2018)
- [Constraint-based Causal Discovery from Multiple Interventions over Overlapping Variable Sets](https://mlsanity.com/paper/1403.2150/) (2014)
- [Strong Completeness and Faithfulness in Bayesian Networks](https://mlsanity.com/paper/1302.4973/) (2013)
- [DoWhy-GCM: An extension of DoWhy for causal inference in graphical causal models](https://mlsanity.com/paper/2206.06821/) (2022)
- [Distinguishing cause from effect using observational data: methods and benchmarks](https://mlsanity.com/paper/1412.3773/) (2014)
- [The Case for Evaluating Causal Models Using Interventional Measures and Empirical Data](https://mlsanity.com/paper/1910.05387/) (2019)
- [Failures of model-dependent generalization bounds for least-norm interpolation](https://mlsanity.com/paper/2010.08479/) (2020)
- [Modeling Causal Mechanisms with Diffusion Models for Interventional and Counterfactual Queries](https://mlsanity.com/paper/2302.00860/) (2023)
- [Causal Inference Through the Structural Causal Marginal Problem](https://mlsanity.com/paper/2202.01300/) (2022)
- [Telling Cause from Effect using MDL-based Local and Global Regression](https://mlsanity.com/paper/1709.08915/) (2017)
- [Phenomenological Causality](https://mlsanity.com/paper/2211.09024/) (2022)

## Built on by (most cited first, 4 of 4)

- [Self-Compatibility: Evaluating Causal Discovery without Ground Truth](https://mlsanity.com/paper/2307.09552/) (2023)
- [On Different Notions of Redundancy in Conditional-Independence-Based Discovery of Graphical Models](https://mlsanity.com/paper/2502.08531/) (2025)
- [Causal Modeling in Multi-Context Systems: Distinguishing Multiple Context-Specific Causal Graphs which Account for Observational Support](https://mlsanity.com/paper/2410.20405/) (2024)
- [Symmetries and Causality: Causal Effect Identification Beyond IID Data](https://mlsanity.com/paper/2609.03697/) (2026)

## Abstract (arXiv)

If X,Y,Z denote sets of random variables, two different data sources may contain samples from PX,Y and PY,Z, respectively. We argue that causal discovery can help inferring properties of the `unobserved joint distributions' PX,Y,Z or PX,Z. The properties may be conditional independences (as in `integrative causal inference') or also quantitative statements about dependences. More generally, we define a learning scenario where the input is a subset of variables and the label is some statistical property of that subset. Sets of jointly observed variables define the training points, while unobserved sets are possible test points. To solve this learning task, we infer, as an intermediate step, a causal model from the observations that then entails properties of unobserved sets. Accordingly, we can define the VC dimension of a class of causal models and derive generalization bounds for the predictions. Here, causal discovery becomes more modest and better accessible to empirical tests than usual: rather than trying to find a causal hypothesis that is `true' a causal hypothesis is it useful whenever it correctly predicts statistical properties of unobserved joint distributions. This way, a sparse causal graph that omits weak influences may be more useful than a dense one (despite being less accurate) because it is able to reconstruct the full joint distribution from marginal distributions of smaller subsets. Within such a `pragmatic' application of causal discovery, some popular heuristic approaches become justified in retrospect. It is, for instance, allowed to infer DAGs from partial correlations instead of conditional independences if the DAGs are only used to predict partial correlations.