# Notes on the Mathematical Structure of GPT LLM Architectures

arXiv: https://arxiv.org/abs/2410.19370 · HTML: https://mlsanity.com/paper/2410.19370/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 1 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Builds on (most cited first, 1 of 1)

- [Formal Algorithms for Transformers](https://mlsanity.com/paper/2207.09238/) (2022)

## Built on by (most cited first, 1 of 1)

- [Hebbian learning the local structure of language](https://mlsanity.com/paper/2503.02057/) (2025)

## Abstract (arXiv)

An exposition of the mathematics underpinning the neural network architecture of a GPT-3-style LLM.