# Mechanisms of Symbol Processing for In-Context Learning in Transformer Networks

arXiv: https://arxiv.org/abs/2410.17498 · HTML: https://mlsanity.com/paper/2410.17498/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2024.

## Built on by (most cited first, 6 of 6)

- [Unlocking Out-of-Distribution Generalization in Transformers via Recursive Latent Space Reasoning](https://mlsanity.com/paper/2510.14095/) (2025)
- [TRA: Better Length Generalisation with Threshold Relative Attention](https://mlsanity.com/paper/2503.23174/) (2025)
- [Tracr-Injection: Distilling Algorithms into Pre-trained Language Models](https://mlsanity.com/paper/2505.10719/) (2025)
- [Kronecker Factorization Improves Efficiency and Interpretability of Sparse Autoencoders](https://mlsanity.com/paper/2505.22255/) (2025)
- [A PyTorch Library of Turing-Complete Neural Networks](https://mlsanity.com/paper/2605.08150/) (2026)
- [Cognitive Convergence: Deep Similarities Between Large Language Models and Human Cognition](https://mlsanity.com/paper/2607.26179/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive abilities in symbol processing through in-context learning (ICL). This success flies in the face of decades of critiques asserting that artificial neural networks cannot master abstract symbol manipulation. We seek to understand the mechanisms that can enable robust symbol processing in transformer networks, illuminating both the unanticipated success, and the significant limitations, of transformers in symbol processing. Borrowing insights from symbolic AI and cognitive science on the power of Production System architectures, we develop a high-level Production System Language, PSL, that allows us to write symbolic programs to do complex, abstract symbol processing, and create compilers that precisely implement PSL programs in transformer networks which are, by construction, 100% mechanistically interpretable. The work is driven by study of a purely abstract (semantics-free) symbolic task that we develop, Templatic Generation (TGT). Although developed through study of TGT, PSL is, we demonstrate, highly general: it is Turing Universal. The new type of transformer architecture that we compile from PSL programs suggests a number of paths for enhancing transformers' capabilities at symbol processing. We note, however, that the work we report addresses computability, and not learnability, by transformer networks. Note: The first section provides an extended synopsis of the entire paper.