# Quantitative probing: Validating causal models using quantitative domain knowledge

arXiv: https://arxiv.org/abs/2209.03013 · HTML: https://mlsanity.com/paper/2209.03013/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 7 Sep 2022.

## Builds on (most cited first, 10 of 10)

- [D'ya like DAGs? A Survey on Structure Learning and Causal Discovery](https://mlsanity.com/paper/2103.02582/) (2021)
- [Causal Inference and Causal Explanation with Background Knowledge](https://mlsanity.com/paper/1302.4972/) (2013)
- [Causal Structure Learning](https://mlsanity.com/paper/1706.09141/) (2017)
- [Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding](https://mlsanity.com/paper/2103.04850/) (2021)
- [RealCause: Realistic Causal Inference Benchmarking](https://mlsanity.com/paper/2011.15007/) (2020)
- [Sense and Sensitivity Analysis: Simple Post-Hoc Analysis of Bias Due to Unobserved Confounding](https://mlsanity.com/paper/2003.01747/) (2020)
- [Long Story Short: Omitted Variable Bias in Causal Machine Learning](https://mlsanity.com/paper/2112.13398/) (2021)
- [A Bayesian Approach to Learning Causal Networks](https://mlsanity.com/paper/1302.4958/) (2013)
- [DoWhy: Addressing Challenges in Expressing and Validating Causal Assumptions](https://mlsanity.com/paper/2108.13518/) (2021)
- [Stable discovery of interpretable subgroups via calibration in causal studies](https://mlsanity.com/paper/2008.10109/) (2020)

## Built on by (most cited first, 6 of 6)

- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Robustness of Algorithms for Causal Structure Learning to Hyperparameter Choice](https://mlsanity.com/paper/2310.18212/) (2023)
- [How PC-based Methods Err: Towards Better Reporting of Assumption Violations and Small Sample Errors](https://mlsanity.com/paper/2502.14719/) (2025)
- [CausalOps -- Towards an Industrial Lifecycle for Causal Probabilistic Graphical Models](https://mlsanity.com/paper/2308.01375/) (2023)
- [Data Driven Diagnosis for Large Cyber-Physical-Systems with Minimal Prior Information](https://mlsanity.com/paper/2506.10613/) (2025)
- [A tutorial on discovering and quantifying the effect of latent causal sources of multimodal EHR data](https://mlsanity.com/paper/2510.16026/) (2025)

## Abstract (arXiv)

We present quantitative probing as a model-agnostic framework for validating causal models in the presence of quantitative domain knowledge. The method is constructed as an analogue of the train/test split in correlation-based machine learning and as an enhancement of current causal validation strategies that are consistent with the logic of scientific discovery. The effectiveness of the method is illustrated using Pearl's sprinkler example, before a thorough simulation-based investigation is conducted. Limits of the technique are identified by studying exemplary failing scenarios, which are furthermore used to propose a list of topics for future research and improvements of the presented version of quantitative probing. The code for integrating quantitative probing into causal analysis, as well as the code for the presented simulation-based studies of the effectiveness of quantitative probing is provided in two separate open-source Python packages.