# Distilling Symbolic Priors for Concept Learning into Neural Networks

arXiv: https://arxiv.org/abs/2402.07035 · HTML: https://mlsanity.com/paper/2402.07035/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 10 Feb 2024.

## Builds on (most cited first, 15 of 15)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [Compositional generalization through meta sequence-to-sequence learning](https://mlsanity.com/paper/1906.05381/) (2019)
- [Modeling rapid language learning by distilling Bayesian priors into artificial neural networks](https://mlsanity.com/paper/2305.14701/) (2023)
- [Meta-learning in natural and artificial intelligence](https://mlsanity.com/paper/2011.13464/) (2020)
- [Universal linguistic inductive biases via meta-learning](https://mlsanity.com/paper/2006.16324/) (2020)
- [People infer recursive visual concepts from just a few examples](https://mlsanity.com/paper/1904.08034/) (2019)
- [One Model for the Learning of Language](https://mlsanity.com/paper/1711.06301/) (2017)
- [Learning Curves for Analysis of Deep Networks](https://mlsanity.com/paper/2010.11029/) (2020)

## Built on by (most cited first, 5 of 5)

- [Whither symbols in the era of advanced neural networks?](https://mlsanity.com/paper/2508.05776/) (2025)
- [Teasing Apart Architecture and Initial Weights as Sources of Inductive Bias in Neural Networks](https://mlsanity.com/paper/2502.20237/) (2025)
- [Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research](https://mlsanity.com/paper/2509.22831/) (2025)
- [Overcoming classic challenges for artificial neural networks by providing incentives and practice](https://mlsanity.com/paper/2410.10596/) (2024)
- [Convolutional Neural Networks Can (Meta-)Learn the Same-Different Relation](https://mlsanity.com/paper/2503.23212/) (2025)

## Abstract (arXiv)

Humans can learn new concepts from a small number of examples by drawing on their inductive biases. These inductive biases have previously been captured by using Bayesian models defined over symbolic hypothesis spaces. Is it possible to create a neural network that displays the same inductive biases? We show that inductive biases that enable rapid concept learning can be instantiated in artificial neural networks by distilling a prior distribution from a symbolic Bayesian model via meta-learning, an approach for extracting the common structure from a set of tasks. By generating the set of tasks used in meta-learning from the prior distribution of a Bayesian model, we are able to transfer that prior into a neural network. We use this approach to create a neural network with an inductive bias towards concepts expressed as short logical formulas. Analyzing results from previous behavioral experiments in which people learned logical concepts from a few examples, we find that our meta-trained models are highly aligned with human performance.