# Language-Based Causal Representation Learning

arXiv: https://arxiv.org/abs/2207.05259 · HTML: https://mlsanity.com/paper/2207.05259/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 58 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 28 of 28)

- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning](https://mlsanity.com/paper/1810.08272/) (2018)
- [Recurrent Independent Mechanisms](https://mlsanity.com/paper/1909.10893/) (2019)
- [Logic Tensor Networks](https://mlsanity.com/paper/2012.13635/) (2020)
- [Invariant Causal Prediction for Block MDPs](https://mlsanity.com/paper/2003.06016/) (2020)
- [The Logic of Graph Neural Networks](https://mlsanity.com/paper/2104.14624/) (2021)
- [Neural Production Systems: Learning Rule-Governed Visual Dynamics](https://mlsanity.com/paper/2103.01937/) (2021)
- [Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundary](https://mlsanity.com/paper/1705.00154/) (2017)
- [Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systems](https://mlsanity.com/paper/2006.16225/) (2020)
- [Leveraging Language to Learn Program Abstractions and Search Heuristics](https://mlsanity.com/paper/2106.11053/) (2021)
- [ASNets: Deep Learning for Generalised Planning](https://mlsanity.com/paper/1908.01362/) (2019)
- [From Statistical to Causal Learning](https://mlsanity.com/paper/2204.00607/) (2022)
- [Learning Generalized Reactive Policies using Deep Neural Networks](https://mlsanity.com/paper/1708.07280/) (2017)
- [Learning First-Order Symbolic Representations for Planning from the Structure of the State Space](https://mlsanity.com/paper/1909.05546/) (2019)
- [Learning General Optimal Policies with Graph Neural Networks: Expressive Power, Transparency, and Limits](https://mlsanity.com/paper/2109.10129/) (2021)
- [Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning](https://mlsanity.com/paper/2006.01096/) (2020)
- [What Can Learned Intrinsic Rewards Capture?](https://mlsanity.com/paper/1912.05500/) (2019)
- [Constraint-based Causal Discovery for Non-Linear Structural Causal Models with Cycles and Latent Confounders](https://mlsanity.com/paper/1807.03024/) (2018)
- [An Explicitly Relational Neural Network Architecture](https://mlsanity.com/paper/1905.10307/) (2019)
- [Making sense of sensory input](https://mlsanity.com/paper/1910.02227/) (2019)
- [Features, Projections, and Representation Change for Generalized Planning](https://mlsanity.com/paper/1801.10055/) (2018)
- [Learning First-Order Representations for Planning from Black-Box States: New Results](https://mlsanity.com/paper/2105.10830/) (2021)
- [Learning Sketches for Decomposing Planning Problems into Subproblems of Bounded Width: Extended Version](https://mlsanity.com/paper/2203.14852/) (2022)
- [Expressing and Exploiting the Common Subgoal Structure of Classical Planning Domains Using Sketches: Extended Version](https://mlsanity.com/paper/2105.04250/) (2021)
- [SLASH: Embracing Probabilistic Circuits into Neural Answer Set Programming](https://mlsanity.com/paper/2110.03395/) (2021)
- [Target Languages (vs. Inductive Biases) for Learning to Act and Plan](https://mlsanity.com/paper/2109.07195/) (2021)
- [Learning First-Order Symbolic Planning Representations That Are Grounded](https://mlsanity.com/paper/2204.11902/) (2022)
- [Learning Generalizable Behavior via Visual Rewrite Rules](https://mlsanity.com/paper/2112.05218/) (2021)

## Abstract (arXiv)

Consider the finite state graph that results from a simple, discrete, dynamical system in which an agent moves in a rectangular grid picking up and dropping packages. Can the state variables of the problem, namely, the agent location and the package locations, be recovered from the structure of the state graph alone without having access to information about the objects, the structure of the states, or any background knowledge? We show that this is possible provided that the dynamics is learned over a suitable domain-independent first-order causal language that makes room for objects and relations that are not assumed to be known. The preference for the most compact representation in the language that is compatible with the data provides a strong and meaningful learning bias that makes this possible. The language of structured causal models (SCMs) is the standard language for representing (static) causal models but in dynamic worlds populated by objects, first-order causal languages such as those used in "classical AI planning" are required. While "classical AI" requires handcrafted representations, similar representations can be learned from unstructured data over the same languages. Indeed, it is the languages and the preference for compact representations in those languages that provide structure to the world, uncovering objects, relations, and causes.