# Graph schemas as abstractions for transfer learning, inference, and planning

arXiv: https://arxiv.org/abs/2302.07350 · HTML: https://mlsanity.com/paper/2302.07350/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 14 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables](https://mlsanity.com/paper/1903.08254/) (2019)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [Search on the Replay Buffer: Bridging Planning and Reinforcement Learning](https://mlsanity.com/paper/1906.05253/) (2019)
- [Assessing Generalization in Deep Reinforcement Learning](https://mlsanity.com/paper/1810.12282/) (2018)
- [Deep Variational Reinforcement Learning for POMDPs](https://mlsanity.com/paper/1806.02426/) (2018)
- [Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs](https://mlsanity.com/paper/2110.05038/) (2021)
- [Abstraction and Analogy-Making in Artificial Intelligence](https://mlsanity.com/paper/2102.10717/) (2021)
- [DeepMind Lab](https://mlsanity.com/paper/1612.03801/) (2016)
- [World Model as a Graph: Learning Latent Landmarks for Planning](https://mlsanity.com/paper/2011.12491/) (2020)
- [Shaping Belief States with Generative Environment Models for RL](https://mlsanity.com/paper/1906.09237/) (2019)
- [Relating transformers to models and neural representations of the hippocampal formation](https://mlsanity.com/paper/2112.04035/) (2021)
- [Emergent Symbols through Binding in External Memory](https://mlsanity.com/paper/2012.14601/) (2020)
- [The StreetLearn Environment and Dataset](https://mlsanity.com/paper/1903.01292/) (2019)
- [Towards mental time travel: a hierarchical memory for reinforcement learning agents](https://mlsanity.com/paper/2105.14039/) (2021)
- [Modeling Perceptual Aliasing in SLAM via Discrete-Continuous Graphical Models](https://mlsanity.com/paper/1810.11692/) (2018)
- [Rapid Task-Solving in Novel Environments](https://mlsanity.com/paper/2006.03662/) (2020)
- [Abstraction for Deep Reinforcement Learning](https://mlsanity.com/paper/2202.05839/) (2022)
- [Representing higher-order dependencies in networks](https://mlsanity.com/paper/1508.03113/) (2015)
- [Learning higher-order sequential structure with cloned HMMs](https://mlsanity.com/paper/1905.00507/) (2019)
- [Learning Overcomplete HMMs](https://mlsanity.com/paper/1711.02309/) (2017)
- [Map Induction: Compositional spatial submap learning for efficient exploration in novel environments](https://mlsanity.com/paper/2110.12301/) (2021)
- [Neural Analogical Matching](https://mlsanity.com/paper/2004.03573/) (2020)
- [Space is a latent sequence: Structured sequence learning as a unified theory of representation in the hippocampus](https://mlsanity.com/paper/2212.01508/) (2022)

## Built on by (most cited first, 7 of 7)

- [Schema-learning and rebinding as mechanisms of in-context learning and emergence](https://mlsanity.com/paper/2307.01201/) (2023)
- [Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environments](https://mlsanity.com/paper/2401.05946/) (2024)
- [Exploring and Learning Structure: Active Inference Approach in Navigational Agents](https://mlsanity.com/paper/2408.05982/) (2024)
- [Schema for In-Context Learning](https://mlsanity.com/paper/2510.13905/) (2025)
- [Learning Dynamic Cognitive Map with Autonomous Navigation](https://mlsanity.com/paper/2411.08447/) (2024)
- [Free Energy Projective Simulation (FEPS): Active inference with interpretability](https://mlsanity.com/paper/2411.14991/) (2024)
- [Learning Successor Features with Distributed Hebbian Temporal Memory](https://mlsanity.com/paper/2310.13391/) (2023)

## Abstract (arXiv)

Transferring latent structure from one environment or problem to another is a mechanism by which humans and animals generalize with very little data. Inspired by cognitive and neurobiological insights, we propose graph schemas as a mechanism of abstraction for transfer learning. Graph schemas start with latent graph learning where perceptually aliased observations are disambiguated in the latent space using contextual information. Latent graph learning is also emerging as a new computational model of the hippocampus to explain map learning and transitive inference. Our insight is that a latent graph can be treated as a flexible template -- a schema -- that models concepts and behaviors, with slots that bind groups of latent nodes to the specific observations or groundings. By treating learned latent graphs (schemas) as prior knowledge, new environments can be quickly learned as compositions of schemas and their newly learned bindings. We evaluate graph schemas on two previously published challenging tasks: the memory & planning game and one-shot StreetLearn, which are designed to test rapid task solving in novel environments. Graph schemas can be learned in far fewer episodes than previous baselines, and can model and plan in a few steps in novel variations of these tasks. We also demonstrate learning, matching, and reusing graph schemas in more challenging 2D and 3D environments with extensive perceptual aliasing and size variations, and show how different schemas can be composed to model larger and more complex environments. To summarize, our main contribution is a unified system, inspired and grounded in cognitive science, that facilitates rapid transfer learning of new environments using schemas via map-induction and composition that handles perceptual aliasing.