# Unifying Causal Inference and Reinforcement Learning using Higher-Order Category Theory

arXiv: https://arxiv.org/abs/2209.06262 · HTML: https://mlsanity.com/paper/2209.06262/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 45 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 7 of 7)

- [Margins of discrete Bayesian networks](https://mlsanity.com/paper/1501.02103/) (2015)
- [An enriched category theory of language: from syntax to semantics](https://mlsanity.com/paper/2106.07890/) (2021)
- [Bipartite Causal Inference with Interference](https://mlsanity.com/paper/1807.08660/) (2018)
- [Universal Decision Models](https://mlsanity.com/paper/2110.15431/) (2021)
- [Generativity and Interactional Effects: an Overview](https://mlsanity.com/paper/1911.10406/) (2019)
- [Causal Inference in Network Economics](https://mlsanity.com/paper/2109.11344/) (2021)
- [On The Universality of Diagrams for Causal Inference and The Causal Reproducing Property](https://mlsanity.com/paper/2207.02917/) (2022)

## Built on by (most cited first, 4 of 4)

- [On the Power of Foundation Models](https://mlsanity.com/paper/2211.16327/) (2022)
- [Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theory](https://mlsanity.com/paper/2310.06756/) (2023)
- [A Layered Architecture for Universal Causality](https://mlsanity.com/paper/2212.08981/) (2022)
- [From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents](https://mlsanity.com/paper/2603.25342/) (2026)

## Abstract (arXiv)

We present a unified formalism for structure discovery of causal models and predictive state representation (PSR) models in reinforcement learning (RL) using higher-order category theory. Specifically, we model structure discovery in both settings using simplicial objects, contravariant functors from the category of ordinal numbers into any category. Fragments of causal models that are equivalent under conditional independence -- defined as causal horns -- as well as subsequences of potential tests in a predictive state representation -- defined as predictive horns -- are both special cases of horns of a simplicial object, subsets resulting from the removal of the interior and the face opposite a particular vertex. Latent structure discovery in both settings involve the same fundamental mathematical problem of finding extensions of horns of simplicial objects through solving lifting problems in commutative diagrams, and exploiting weak homotopies that define higher-order symmetries. Solutions to the problem of filling "inner" vs "outer" horns leads to various notions of higher-order categories, including weak Kan complexes and quasicategories. We define the abstract problem of structure discovery in both settings in terms of adjoint functors between the category of universal causal models or universal decision models and its simplicial object representation.