mlsanityData to 13 Oct 2026

Reconciling Shared versus Context-Specific Information in a Neural Network Model of Latent Causes

arXiv 2312.08519 ↗First posted 13 Dec 2023q-bio.NC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

It has been proposed that, when processing a stream of events, humans divide their experiences in terms of inferred latent causes (LCs) to support context-dependent learning. However, when shared structure is present across contexts, it is still unclear how the "splitting" of LCs and learning of shared structure can be simultaneously achieved. Here, we present the Latent Cause Network (LCNet), a neural network model of LC inference. Through learning, it naturally stores structure that is shared across tasks in the network weights. Additionally, it represents context-specific structure using a context module, controlled by a Bayesian nonparametric inference algorithm, which assigns a unique context vector for each inferred LC. Across three simulations, we found that LCNet could 1) extract shared structure across LCs in a function learning task while avoiding catastrophic interference, 2) capture human data on curriculum effects in schema learning, and 3) infer the underlying event structure when processing naturalistic videos of daily events. Overall, these results demonstrate a computationally feasible approach to reconciling shared structure and context-specific structure in a model of LCs that is scalable from laboratory experiment settings to naturalistic settings.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 18 of its 100 references that are in the map. First posted on arXiv 13 Dec 2023.

Builds on

18 of 100 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,106
  2. 2023GPT-4 Technical Reportcited by 18,278
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,753
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,024
  5. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,322
  6. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 675
  7. 2019Three scenarios for continual learningcited by 612
  8. 2018Progress & Compress: A scalable framework for continual learningcited by 516
  9. 2018Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilizationcited by 104
  10. 2018An analytic theory of generalization dynamics and transfer learning in deep linear networkscited by 85
  11. 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
  12. 2016One-shot Learning with Memory-Augmented Neural Networkscited by 60
6 more
  1. 2018Been There, Done That: Meta-Learning with Episodic Recallcited by 25
  2. 2022Modelling continual learning in humans with Hebbian context gating and exponentially decaying task signalscited by 12
  3. 2022Continual task learning in natural and artificial agentscited by 12
  4. 2022A Neural Network Model of Continual Learning with Cognitive Controlcited by 4
  5. 2022Thalamus: a brain-inspired algorithm for biologically-plausible continual learning and disentangled representationscited by 4
  6. 2022Multi-Task Dynamical Systemscited by 1

Built on by

Not cited yet.