# Neural Network Quantum Field Theory from Transformer Architectures

arXiv: https://arxiv.org/abs/2602.10209 · HTML: https://mlsanity.com/paper/2602.10209/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 10 Feb 2026.

## Builds on (most cited first, 10 of 10)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer](https://mlsanity.com/paper/2203.03466/) (2022)
- [Infinite attention: NNGP and NTK for deep attention networks](https://mlsanity.com/paper/2006.10540/) (2020)
- [Infinite Limits of Multi-head Transformer Dynamics](https://mlsanity.com/paper/2405.15712/) (2024)
- [Transformers are Graph Neural Networks](https://mlsanity.com/paper/2506.22084/) (2025)
- [Building Quantum Field Theories Out of Neurons](https://mlsanity.com/paper/2112.04527/) (2021)
- [Neural Network Field Theories: Non-Gaussianity, Actions, and Locality](https://mlsanity.com/paper/2307.03223/) (2023)
- [Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs](https://mlsanity.com/paper/2506.00846/) (2025)

## Abstract (arXiv)

We propose a neural-network construction of Euclidean scalar quantum field theories from transformer attention heads, defining n-point correlators by averaging over random network parameters in the NN-QFT framework. For a single attention head, shared random softmax weights couple different width coordinates and induce non-Gaussian field statistics that persist in the infinite-width limit dk→∞. We compute the two-point function in an attention-weight representation and show how Euclidean-invariant kernels can be engineered via random-feature token embeddings. We then analyze the connected four-point function and identify an "independence-breaking" contribution, expressible as a covariance over query-key weights, which remains finite at infinite width. Finally, we show that summing many independent heads with standard 1/Nh normalization suppresses connected non-Gaussian correlators as 1/Nh, yielding a Gaussian NN-QFT in the large-head limit.