# The Causal Description Gap: Information-Theoretic Separations Across Pearl's Hierarchy

arXiv: https://arxiv.org/abs/2605.02177 · HTML: https://mlsanity.com/paper/2605.02177/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 4 May 2026.

## Builds on (most cited first, 5 of 5)

- [Causal Reasoning and Large Language Models: Opening a New Frontier for Causality](https://mlsanity.com/paper/2305.00050/) (2023)
- [CLadder: Assessing Causal Reasoning in Language Models](https://mlsanity.com/paper/2312.04350/) (2023)
- [Learning Causal Graphs with Small Interventions](https://mlsanity.com/paper/1511.00041/) (2015)
- [Comparing Causal Frameworks: Potential Outcomes, Structural Models, Graphs, and Abstractions](https://mlsanity.com/paper/2306.14351/) (2023)
- [From Probability to Counterfactuals: the Increasing Complexity of Satisfiability in Pearl's Causal Hierarchy](https://mlsanity.com/paper/2405.07373/) (2024)

## Abstract (arXiv)

Pearl's causal hierarchy shows that observational, interventional, and counterfactual queries are qualitatively distinct. We ask a quantitative version of this question: how many additional bits are needed to specify higher-rung causal answers once lower-rung answers are known? We formalize this via query-class description length, the Kolmogorov complexity of the answer oracle induced by an SCM for a class of queries. Our main construction gives binary acyclic SCMs whose observational distribution has constant description length, while the single-variable interventional answer oracle has description length Theta(n2). A degree-sensitive upper bound shows that finite-gate-schema SCMs of indegree d have observational-interventional gap at most O(nd log(en/d) + n log n), making the quadratic construction order-optimal in the dense regime and a rooted-tree construction order-optimal for bounded indegree. The quadratic separation persists under varepsilon-accurate total-variation descriptions for every fixed varepsilon < 1/4. At the next rung, the full hard-do interventional oracle can still leave a Theta(n) counterfactual description gap. A general ambiguity-to-bits theorem and Shannon analogue show that these gaps equal the logarithm of residual higher-rung ambiguity up to lower-order terms.