# Reconciling Shared versus Context-Specific Information in a Neural Network Model of Latent Causes

arXiv: https://arxiv.org/abs/2312.08519 · HTML: https://mlsanity.com/paper/2312.08519/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 100 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [Three scenarios for continual learning](https://mlsanity.com/paper/1904.07734/) (2019)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization](https://mlsanity.com/paper/1802.01569/) (2018)
- [An analytic theory of generalization dynamics and transfer learning in deep linear networks](https://mlsanity.com/paper/1809.10374/) (2018)
- [Evaluating Cognitive Maps and Planning in Large Language Models with CogEval](https://mlsanity.com/paper/2309.15129/) (2023)
- [One-shot Learning with Memory-Augmented Neural Networks](https://mlsanity.com/paper/1605.06065/) (2016)
- [Been There, Done That: Meta-Learning with Episodic Recall](https://mlsanity.com/paper/1805.09692/) (2018)
- [Modelling continual learning in humans with Hebbian context gating and exponentially decaying task signals](https://mlsanity.com/paper/2203.11560/) (2022)
- [Continual task learning in natural and artificial agents](https://mlsanity.com/paper/2210.04520/) (2022)
- [A Neural Network Model of Continual Learning with Cognitive Control](https://mlsanity.com/paper/2202.04773/) (2022)
- [Thalamus: a brain-inspired algorithm for biologically-plausible continual learning and disentangled representations](https://mlsanity.com/paper/2205.11713/) (2022)
- [Multi-Task Dynamical Systems](https://mlsanity.com/paper/2210.04023/) (2022)

## Abstract (arXiv)

It has been proposed that, when processing a stream of events, humans divide their experiences in terms of inferred latent causes (LCs) to support context-dependent learning. However, when shared structure is present across contexts, it is still unclear how the "splitting" of LCs and learning of shared structure can be simultaneously achieved. Here, we present the Latent Cause Network (LCNet), a neural network model of LC inference. Through learning, it naturally stores structure that is shared across tasks in the network weights. Additionally, it represents context-specific structure using a context module, controlled by a Bayesian nonparametric inference algorithm, which assigns a unique context vector for each inferred LC. Across three simulations, we found that LCNet could 1) extract shared structure across LCs in a function learning task while avoiding catastrophic interference, 2) capture human data on curriculum effects in schema learning, and 3) infer the underlying event structure when processing naturalistic videos of daily events. Overall, these results demonstrate a computationally feasible approach to reconciling shared structure and context-specific structure in a model of LCs that is scalable from laboratory experiment settings to naturalistic settings.