# Conceptual Cognitive Maps Formation with Neural Successor Networks and Word Embeddings

arXiv: https://arxiv.org/abs/2307.01577 · HTML: https://mlsanity.com/paper/2307.01577/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 40 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Builds on (most cited first, 8 of 8)

- [How deep is deep enough? -- Quantifying class separability in the hidden layers of deep neural networks](https://mlsanity.com/paper/1811.01753/) (2018)
- [A statistical method for analyzing and comparing spatiotemporal cortical activation patterns](https://mlsanity.com/paper/1611.07677/) (2016)
- [Analysis of structure and dynamics in three-neuron motifs](https://mlsanity.com/paper/1811.05225/) (2018)
- [Neural Network based Successor Representations of Space and Language](https://mlsanity.com/paper/2202.11190/) (2022)
- [Word class representations spontaneously emerge in a deep neural network trained on next word prediction](https://mlsanity.com/paper/2302.07588/) (2023)
- [Neural Network based Formation of Cognitive Maps of Semantic Spaces and the Emergence of Abstract Concepts](https://mlsanity.com/paper/2210.16062/) (2022)
- [Classification at the Accuracy Limit -- Facing the Problem of Data Ambiguity](https://mlsanity.com/paper/2206.01922/) (2022)
- [Quantifying and maximizing the information flux in recurrent neural networks](https://mlsanity.com/paper/2301.12892/) (2023)

## Built on by (most cited first, 4 of 4)

- [Multi-Modal Cognitive Maps based on Neural Networks trained on Successor Representations](https://mlsanity.com/paper/2401.01364/) (2023)
- [Beyond Labels: Advancing Cluster Analysis with the Entropy of Distance Distribution (EDD)](https://mlsanity.com/paper/2311.16621/) (2023)
- [Probing for Consciousness in Machines](https://mlsanity.com/paper/2411.16262/) (2024)
- [Word-Class and Construction-Like Structure Emerges in Neural Successor Representations Trained on Natural Language](https://mlsanity.com/paper/2605.24585/) (2026)

## Abstract (arXiv)

The human brain possesses the extraordinary capability to contextualize the information it receives from our environment. The entorhinal-hippocampal plays a critical role in this function, as it is deeply engaged in memory processing and constructing cognitive maps using place and grid cells. Comprehending and leveraging this ability could significantly augment the field of artificial intelligence. The multi-scale successor representation serves as a good model for the functionality of place and grid cells and has already shown promise in this role. Here, we introduce a model that employs successor representations and neural networks, along with word embedding vectors, to construct a cognitive map of three separate concepts. The network adeptly learns two different scaled maps and situates new information in proximity to related pre-existing representations. The dispersion of information across the cognitive map varies according to its scale - either being heavily concentrated, resulting in the formation of the three concepts, or spread evenly throughout the map. We suggest that our model could potentially improve current AI models by providing multi-modal context information to any input, based on a similarity metric for the input and pre-existing knowledge representations.