# The Emergent Symbolic Structure of Artificial Neural Networks

arXiv: https://arxiv.org/abs/2608.29530 · HTML: https://mlsanity.com/paper/2608.29530/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 183 references that are in the map.
- First posted on arXiv 30 Aug 2026.

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [gpt-oss-120b & gpt-oss-20b Model Card](https://mlsanity.com/paper/2508.10925/) (2025)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI](https://mlsanity.com/paper/2411.04872/) (2024)
- [On the Binding Problem in Artificial Neural Networks](https://mlsanity.com/paper/2012.05208/) (2020)
- [A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part I: Models and Data Transformations](https://mlsanity.com/paper/2111.06077/) (2021)
- [Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving](https://mlsanity.com/paper/1910.06611/) (2019)
- [Do Sparse Autoencoders Capture Concept Manifolds?](https://mlsanity.com/paper/2604.28119/) (2026)
- [Symbolic Behaviour in Artificial Intelligence](https://mlsanity.com/paper/2102.03406/) (2021)
- [GPT-2 Through the Lens of Vector Symbolic Architectures](https://mlsanity.com/paper/2412.07947/) (2024)
- [Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic Architectures](https://mlsanity.com/paper/2509.25045/) (2025)
- [Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics](https://mlsanity.com/paper/2606.06533/) (2026)
- [Optimal quadratic binding for relational reasoning in vector symbolic neural architectures](https://mlsanity.com/paper/2204.07186/) (2022)
- [Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions](https://mlsanity.com/paper/2605.09967/) (2026)
- [Linear representations of grammaticality in neural language models](https://mlsanity.com/paper/2607.15175/) (2026)

## Built on by (most cited first, 1 of 1)

- [Binding Phoneme Identity to Position in Neural Models and the Human Brain](https://mlsanity.com/paper/2610.15843/) (2026)

## Abstract (arXiv)

Modern systems in artificial intelligence (AI) somehow excel in domains for which they seem poorly suited. Intelligence has traditionally been modeled as operating over structured combinations of symbols, such as logical formulas. However, the strongest modern AI systems are based on neural networks, which instead represent information in continuous vectors. Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas. How do they do it? In this work, we propose a potential answer: Despite appearances, perhaps the internal representations of neural networks implicitly realize symbolic structure. In support of this hypothesis, we show that the vector representations of a variety of neural networks can be closely approximated with symbolic structures: we can replace the network's entire representation-generating process with a closed-form equation instantiating a symbolic structure, and the network's behavior remains largely unchanged. This finding holds for both small-scale neural networks trained to manipulate lists as well as large language models (LLMs) operating in four domains that are central in symbolic traditions: arithmetic, logic, computer code, and language. Further, our symbolic approximation allows us to modify an LLM's behavior in targeted ways via precise interventions on its internal representations, showing that the LLM's behavior is reliant on the symbolic structures we have identified. This work provides a potential way to reconcile longstanding symbolic conceptions of intelligence with the vector-based nature of modern AI.