# Is attention all you need to solve the correlated electron problem?

arXiv: https://arxiv.org/abs/2502.05383 · HTML: https://mlsanity.com/paper/2502.05383/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 71 references that are in the map.
- First posted on arXiv 7 Feb 2025.

## Builds on (most cited first, 12 of 12)

- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [SchNet: A continuous-filter convolutional neural network for modeling quantum interactions](https://mlsanity.com/paper/1706.08566/) (2017)
- [Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networks](https://mlsanity.com/paper/1909.02487/) (2019)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [A Self-Attention Ansatz for Ab-initio Quantum Chemistry](https://mlsanity.com/paper/2211.13672/) (2022)
- [Generalizing Neural Wave Functions](https://mlsanity.com/paper/2302.04168/) (2023)
- [Discovering Quantum Phase Transitions with Fermionic Neural Networks](https://mlsanity.com/paper/2202.05183/) (2022)
- [Gauge Invariant and Anyonic Symmetric Transformer and RNN Quantum States for Quantum Lattice Models](https://mlsanity.com/paper/2101.07243/) (2021)
- [Accurate Computation of Quantum Excited States with Neural Networks](https://mlsanity.com/paper/2308.16848/) (2023)
- [Pairing-based graph neural network for simulating quantum materials](https://mlsanity.com/paper/2311.02143/) (2023)
- [Graph-Aware Isomorphic Attention for Adaptive Dynamics in Transformers](https://mlsanity.com/paper/2501.02393/) (2025)
- [Forward Laplacian: A New Computational Framework for Neural Network-based Variational Monte Carlo](https://mlsanity.com/paper/2307.08214/) (2023)

## Built on by (most cited first, 2 of 2)

- [Reduced Density Matrices Through Machine Learning](https://mlsanity.com/paper/2511.07367/) (2025)
- [Representability-Aware Neural Networks for Reduced Density Matrices: Application to Fractional Chern Insulators](https://mlsanity.com/paper/2605.20326/) (2026)

## Abstract (arXiv)

The attention mechanism has transformed artificial intelligence research by its ability to learn relations between objects. In this work, we explore how a many-body wavefunction ansatz constructed from a large-parameter self-attention neural network can be used to solve the interacting electron problem in solids. By a systematic neural-network variational Monte Carlo study on a moir'e quantum material, we demonstrate that the self-attention ansatz provides an accurate and efficient solution without human bias. Moreover, our numerical study finds that the required number of variational parameters scales roughly as N2 with the number of electrons, which opens a path towards efficient large-scale simulations.