# On the Interventional Kullback-Leibler Divergence

arXiv: https://arxiv.org/abs/2302.05380 · HTML: https://mlsanity.com/paper/2302.05380/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 10 Feb 2023.

## Builds on (most cited first, 14 of 14)

- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style](https://mlsanity.com/paper/2106.04619/) (2021)
- [Towards Causal Representation Learning](https://mlsanity.com/paper/2102.11107/) (2021)
- [Causal Discovery from Heterogeneous/Nonstationary Data with Independent Changes](https://mlsanity.com/paper/1903.01672/) (2019)
- [Probable Domain Generalization via Quantile Risk Minimization](https://mlsanity.com/paper/2207.09944/) (2022)
- [Causal Discovery in Heterogeneous Environments Under the Sparse Mechanism Shift Hypothesis](https://mlsanity.com/paper/2206.02013/) (2022)
- [Causal de Finetti: On the Identification of Invariant Causal Structure in Exchangeable Data](https://mlsanity.com/paper/2203.15756/) (2022)
- [Why did the distribution change?](https://mlsanity.com/paper/2102.13384/) (2021)
- [Learning and Testing Causal Models with Interventions](https://mlsanity.com/paper/1805.09697/) (2018)
- [Jointly interventional and observational data: estimation of interventional Markov equivalence classes of directed acyclic graphs](https://mlsanity.com/paper/1303.3216/) (2013)
- [Structural Intervention Distance (SID) for Evaluating Causal Graphs](https://mlsanity.com/paper/1306.1043/) (2013)
- [A Ladder of Causal Distances](https://mlsanity.com/paper/2005.02480/) (2020)
- [Causal Network Learning from Multiple Interventions of Unknown Manipulated Targets](https://mlsanity.com/paper/1610.08611/) (2016)

## Built on by (most cited first, 5 of 5)

- [Scalable Counterfactual Distribution Estimation in Multivariate Causal Models](https://mlsanity.com/paper/2311.00927/) (2023)
- [Out-of-Variable Generalization for Discriminative Models](https://mlsanity.com/paper/2304.07896/) (2023)
- [Algorithmic causal structure emerging through compression](https://mlsanity.com/paper/2502.04210/) (2025)
- [Beyond Single-Feature Importance with ICECREAM](https://mlsanity.com/paper/2307.09779/) (2023)
- [Disentangled Representations for Causal Cognition](https://mlsanity.com/paper/2407.00744/) (2024)

## Abstract (arXiv)

Modern machine learning approaches excel in static settings where a large amount of i.i.d. training data are available for a given task. In a dynamic environment, though, an intelligent agent needs to be able to transfer knowledge and re-use learned components across domains. It has been argued that this may be possible through causal models, aiming to mirror the modularity of the real world in terms of independent causal mechanisms. However, the true causal structure underlying a given set of data is generally not identifiable, so it is desirable to have means to quantify differences between models (e.g., between the ground truth and an estimate), on both the observational and interventional level. In the present work, we introduce the Interventional Kullback-Leibler (IKL) divergence to quantify both structural and distributional differences between models based on a finite set of multi-environment distributions generated by interventions from the ground truth. Since we generally cannot quantify all differences between causal models for every finite set of interventional distributions, we propose a sufficient condition on the intervention targets to identify subsets of observed variables on which the models provably agree or disagree.