# Relational reasoning and generalization using non-symbolic neural networks

arXiv: https://arxiv.org/abs/2006.07968 · HTML: https://mlsanity.com/paper/2006.07968/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2020.

## Built on by (most cited first, 15 of 15)

- [Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations](https://mlsanity.com/paper/2303.02536/) (2023)
- [Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability](https://mlsanity.com/paper/2301.04709/) (2023)
- [Tell me why! Explanations support learning relational and causal structure](https://mlsanity.com/paper/2112.03753/) (2021)
- [How Do Transformers Learn Variable Binding in Symbolic Programs?](https://mlsanity.com/paper/2505.20896/) (2025)
- [MLPs Learn In-Context on Regression and Classification Tasks](https://mlsanity.com/paper/2405.15618/) (2024)
- [Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects](https://mlsanity.com/paper/2406.15955/) (2024)
- [How Causal Abstraction Underpins Computational Explanation](https://mlsanity.com/paper/2508.11214/) (2025)
- [Does Deep Learning Learn to Abstract? A Systematic Probing Framework](https://mlsanity.com/paper/2302.11978/) (2023)
- [Learning the meanings of function words from grounded language using a visual question answering model](https://mlsanity.com/paper/2308.08628/) (2023)
- [Philosophy of Cognitive Science in the Age of Deep Learning](https://mlsanity.com/paper/2405.04048/) (2024)
- [GPT-ology, Computational Models, Silicon Sampling: How should we think about LLMs in Cognitive Science?](https://mlsanity.com/paper/2406.09464/) (2024)
- [Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations](https://mlsanity.com/paper/2310.09612/) (2023)
- [Systematic human learning and generalization from a brief tutorial with explanatory feedback](https://mlsanity.com/paper/2107.06994/) (2021)
- [Learning richness modulates equality reasoning in neural networks](https://mlsanity.com/paper/2503.09781/) (2025)
- [On the use of foundation models in cognitive science](https://mlsanity.com/paper/2608.07812/) (2026)

## Abstract (arXiv)

The notion of equality (identity) is simple and ubiquitous, making it a key case study for broader questions about the representations supporting abstract relational reasoning. Previous work suggested that neural networks were not suitable models of human relational reasoning because they could not represent mathematically identity, the most basic form of equality. We revisit this question. In our experiments, we assess out-of-sample generalization of equality using both arbitrary representations and representations that have been pretrained on separate tasks to imbue them with structure. We find neural networks are able to learn (1) basic equality (mathematical identity), (2) sequential equality problems (learning ABA-patterned sequences) with only positive training instances, and (3) a complex, hierarchical equality problem with only basic equality training instances ("zero-shot'" generalization). In the two latter cases, our models perform tasks proposed in previous work to demarcate human-unique symbolic abilities. These results suggest that essential aspects of symbolic reasoning can emerge from data-driven, non-symbolic learning processes.