# Gauge Invariant and Anyonic Symmetric Transformer and RNN Quantum States for Quantum Lattice Models

arXiv: https://arxiv.org/abs/2101.07243 · HTML: https://mlsanity.com/paper/2101.07243/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 80 references that are in the map.
- First posted on arXiv 18 Jan 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networks](https://mlsanity.com/paper/1909.02487/) (2019)
- [Deep neural network solution of the electronic Schr"odinger equation](https://mlsanity.com/paper/1909.08423/) (2019)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [Equivariant flow-based sampling for lattice gauge theory](https://mlsanity.com/paper/2003.06413/) (2020)
- [Sampling using SU(N) gauge equivariant flows](https://mlsanity.com/paper/2008.05456/) (2020)
- [Deep autoregressive models for the efficient variational simulation of many-body quantum systems](https://mlsanity.com/paper/1902.04057/) (2019)
- [Quantum Entanglement in Deep Learning Architectures](https://mlsanity.com/paper/1803.09780/) (2018)
- [Neural-Network Quantum States, String-Bond States, and Chiral Topological States](https://mlsanity.com/paper/1710.04045/) (2017)
- [Gauge equivariant neural networks for quantum lattice gauge theories](https://mlsanity.com/paper/2012.05232/) (2020)
- [Phases of two-dimensional spinless lattice fermions with first-quantized deep neural-network quantum states](https://mlsanity.com/paper/2008.00118/) (2020)
- [Watch and learn -- a generalized approach for transferrable learning in deep neural networks via physical principles](https://mlsanity.com/paper/2003.02647/) (2020)

## Built on by (most cited first, 18 of 18)

- [Diffusion Models as Stochastic Quantization in Lattice Field Theory](https://mlsanity.com/paper/2309.17082/) (2023)
- [QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptation](https://mlsanity.com/paper/2406.00132/) (2024)
- [L2M: Mutual Information Scaling Law for Long-Context Language Modeling](https://mlsanity.com/paper/2503.04725/) (2025)
- [Recurrent neural network wave functions for Rydberg atom arrays on kagome lattice](https://mlsanity.com/paper/2405.20384/) (2024)
- [Are queries and keys always relevant? A case study on Transformer wave functions](https://mlsanity.com/paper/2405.18874/) (2024)
- [Approximately-symmetric neural networks for quantum spin liquids](https://mlsanity.com/paper/2405.17541/) (2024)
- [TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision](https://mlsanity.com/paper/2404.10771/) (2024)
- [Gauge-Equivariant Graph Neural Networks for Lattice Gauge Theories](https://mlsanity.com/paper/2604.20797/) (2026)
- [Is attention all you need to solve the correlated electron problem?](https://mlsanity.com/paper/2502.05383/) (2025)
- [Graph Neural Networks in the Wilson Loop Representation of Abelian Lattice Gauge Theories](https://mlsanity.com/paper/2605.03901/) (2026)
- [Parallel Scan Recurrent Neural Quantum States for Scalable Variational Monte Carlo](https://mlsanity.com/paper/2605.13807/) (2026)
- [Learning quantum symmetries with interactive quantum-classical variational algorithms](https://mlsanity.com/paper/2206.11970/) (2022)
- [Diagonal Symmetrization of Neural Network Solvers for the Many-Electron Schr"odinger Equation](https://mlsanity.com/paper/2502.05318/) (2025)
- [Adaptive Neural Quantum States: A Recurrent Neural Network Perspective](https://mlsanity.com/paper/2507.18700/) (2025)
- [Geometry-Induced Long-Range Correlations in Recurrent Neural Network Quantum States](https://mlsanity.com/paper/2604.08661/) (2026)
- [Mechanistic Interpretability and Causal Feature Steering of Neural Quantum States via Sparse Autoencoders](https://mlsanity.com/paper/2607.01336/) (2026)
- [TENG-BC: Unified Time-Evolving Natural Gradient for Neural PDE Solvers with General Boundary Conditions](https://mlsanity.com/paper/2603.00397/) (2026)
- [Emergent Loop Geometry in Gauge-Equivariant Graph Neural Networks](https://mlsanity.com/paper/2610.18742/) (2026)

## Abstract (arXiv)

Symmetries such as gauge invariance and anyonic symmetry play a crucial role in quantum many-body physics. We develop a general approach to constructing gauge invariant or anyonic symmetric autoregressive neural network quantum states, including a wide range of architectures such as Transformer and recurrent neural network (RNN), for quantum lattice models. These networks can be efficiently sampled and explicitly obey gauge symmetries or anyonic constraint. We prove that our methods can provide exact representation for the ground and excited states of the 2D and 3D toric codes, and the X-cube fracton model. We variationally optimize our symmetry incorporated autoregressive neural networks for ground states as well as real-time dynamics for a variety of models. We simulate the dynamics and the ground states of the quantum link model of U(1) lattice gauge theory, obtain the phase diagram for the 2D mathbbZ2 gauge theory, determine the phase transition and the central charge of the SU(2)3 anyonic chain, and also compute the ground state energy of the SU(2) invariant Heisenberg spin chain. Our approach provides powerful tools for exploring condensed matter physics, high energy physics and quantum information science.