# NNQS-Transformer: an Efficient and Scalable Neural Network Quantum States Approach for Ab initio Quantum Chemistry

arXiv: https://arxiv.org/abs/2306.16705 · HTML: https://mlsanity.com/paper/2306.16705/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [MADE: Masked Autoencoder for Distribution Estimation](https://mlsanity.com/paper/1502.03509/) (2015)
- [Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networks](https://mlsanity.com/paper/1909.02487/) (2019)
- [Deep neural network solution of the electronic Schr"odinger equation](https://mlsanity.com/paper/1909.08423/) (2019)
- [TurboTransformers: An Efficient GPU Serving System For Transformer Models](https://mlsanity.com/paper/2010.05680/) (2020)
- [Deep autoregressive models for the efficient variational simulation of many-body quantum systems](https://mlsanity.com/paper/1902.04057/) (2019)
- [Neural-Network Quantum States, String-Bond States, and Chiral Topological States](https://mlsanity.com/paper/1710.04045/) (2017)
- [Neural tensor contractions and the expressive power of deep neural quantum states](https://mlsanity.com/paper/2103.10293/) (2021)
- [Autoregressive neural-network wavefunctions for ab initio quantum chemistry](https://mlsanity.com/paper/2109.12606/) (2021)
- [LightSeq2: Accelerated Training for Transformer-based Models on GPUs](https://mlsanity.com/paper/2110.05722/) (2021)
- [Overcoming barriers to scalability in variational quantum Monte Carlo](https://mlsanity.com/paper/2106.13308/) (2021)
- [Scalable neural quantum states architecture for quantum chemistry](https://mlsanity.com/paper/2208.05637/) (2022)

## Built on by (most cited first, 8 of 8)

- [Retentive Neural Quantum States: Efficient Ans"atze for Ab Initio Quantum Chemistry](https://mlsanity.com/paper/2411.03900/) (2024)
- [Large Language Model Scaling Laws for Neural Quantum States in Quantum Chemistry](https://mlsanity.com/paper/2509.12679/) (2025)
- [A Fully GPU-Accelerated Framework for High-Performance Configuration Interaction Selection with Neural Network Quantum States](https://mlsanity.com/paper/2604.15768/) (2026)
- [Variational quantum and neural quantum states algorithms for the linear complementarity problem](https://mlsanity.com/paper/2504.08141/) (2025)
- [Enhanced Neural Quantum State via Annealed Gradient Descent](https://mlsanity.com/paper/2607.18865/) (2026)
- [Foundation Neural-Network Quantum States for Molecular Potential Energy Surfaces in Second Quantization](https://mlsanity.com/paper/2609.37733/) (2026)
- [Neural Fourier Surrogates for Data Reuploading Quantum Neural Networks](https://mlsanity.com/paper/2610.00841/) (2026)
- [Generalization of Transformer-Based Neural Quantum States via In-Context Learning](https://mlsanity.com/paper/2610.03463/) (2026)

## Abstract (arXiv)

Neural network quantum state (NNQS) has emerged as a promising candidate for quantum many-body problems, but its practical applications are often hindered by the high cost of sampling and local energy calculation. We develop a high-performance NNQS method for textitab initio electronic structure calculations. The major innovations include: (1) A transformer based architecture as the quantum wave function ansatz; (2) A data-centric parallelization scheme for the variational Monte Carlo (VMC) algorithm which preserves data locality and well adapts for different computing architectures; (3) A parallel batch sampling strategy which reduces the sampling cost and achieves good load balance; (4) A parallel local energy evaluation scheme which is both memory and computationally efficient; (5) Study of real chemical systems demonstrates both the superior accuracy of our method compared to state-of-the-art and the strong and weak scalability for large molecular systems with up to 120 spin orbitals.