# Algorithmic causal structure emerging through compression

arXiv: https://arxiv.org/abs/2502.04210 · HTML: https://mlsanity.com/paper/2502.04210/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 49 references that are in the map.
- First posted on arXiv 6 Feb 2025.

## Builds on (most cited first, 12 of 12)

- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [A Theory for Emergence of Complex Skills in Language Models](https://mlsanity.com/paper/2307.15936/) (2023)
- [CLadder: Assessing Causal Reasoning in Language Models](https://mlsanity.com/paper/2312.04350/) (2023)
- [Causal Discovery in Heterogeneous Environments Under the Sparse Mechanism Shift Hypothesis](https://mlsanity.com/paper/2206.02013/) (2022)
- [Context-Specific Independence in Bayesian Networks](https://mlsanity.com/paper/1302.3562/) (2013)
- [Causal Inference Using Tractable Circuits](https://mlsanity.com/paper/2202.02891/) (2022)
- [Bivariate Causal Discovery using Bayesian Model Selection](https://mlsanity.com/paper/2306.02931/) (2023)
- [A Measure-Theoretic Axiomatisation of Causality](https://mlsanity.com/paper/2305.17139/) (2023)
- [Causal Inference on Multivariate and Mixed-Type Data](https://mlsanity.com/paper/1702.06385/) (2017)
- [Causal Discovery using Compression-Complexity Measures](https://mlsanity.com/paper/2010.09336/) (2020)
- [On the Interventional Kullback-Leibler Divergence](https://mlsanity.com/paper/2302.05380/) (2023)
- [Causal Unit Selection using Tractable Arithmetic Circuits](https://mlsanity.com/paper/2404.06681/) (2024)

## Built on by (most cited first, 3 of 3)

- [How Causal Abstraction Underpins Computational Explanation](https://mlsanity.com/paper/2508.11214/) (2025)
- [Learning Causal Orderings for In-Context Tabular Prediction](https://mlsanity.com/paper/2605.22335/) (2026)
- [Dictionary Based Pattern Entropy for Causal Direction Discovery](https://mlsanity.com/paper/2603.04473/) (2026)

## Abstract (arXiv)

We explore the relationship between causality, symmetry, and compression. We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable. We propose a framework where causality emerges as a consequence of compressing data across multiple environments. We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold. We demonstrate how algorithmic causal and symmetric structures can emerge from minimizing upper bounds on Kolmogorov complexity, without knowledge of intervention targets. We hypothesize that these insights may also provide a novel perspective on the emergence of causality in machine learning models, such as large language models, where causal relationships may not be explicitly identifiable.