# Towards a Foundation Model for Neural Network Wavefunctions

arXiv: https://arxiv.org/abs/2303.09949 · HTML: https://mlsanity.com/paper/2303.09949/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 17 Mar 2023.

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networks](https://mlsanity.com/paper/1909.02487/) (2019)
- [Deep neural network solution of the electronic Schr"odinger equation](https://mlsanity.com/paper/1909.08423/) (2019)
- [The Design Space of E(3)-Equivariant Atom-Centered Interatomic Potentials](https://mlsanity.com/paper/2205.06643/) (2022)
- [Unifying machine learning and quantum chemistry -- a deep neural network for molecular wavefunctions](https://mlsanity.com/paper/1906.10033/) (2019)
- [SE(3)-equivariant prediction of molecular wavefunctions and electronic densities](https://mlsanity.com/paper/2106.02347/) (2021)
- [A Self-Attention Ansatz for Ab-initio Quantum Chemistry](https://mlsanity.com/paper/2211.13672/) (2022)
- [Ab-Initio Potential Energy Surfaces by Pairing GNNs with Neural Wave Functions](https://mlsanity.com/paper/2110.05064/) (2021)
- [Machine learning for electronically excited states of molecules](https://mlsanity.com/paper/2007.05320/) (2020)
- [Better, Faster Fermionic Neural Networks](https://mlsanity.com/paper/2011.07125/) (2020)
- [Generalizing Neural Wave Functions](https://mlsanity.com/paper/2302.04168/) (2023)
- [Gold-standard solutions to the Schr"odinger equation using deep learning: How much physics do we need?](https://mlsanity.com/paper/2205.09438/) (2022)
- [Discovering Quantum Phase Transitions with Fermionic Neural Networks](https://mlsanity.com/paper/2202.05183/) (2022)
- [Solving the electronic Schr"odinger equation for multiple nuclear geometries with weight-sharing deep neural networks](https://mlsanity.com/paper/2105.08351/) (2021)
- [Electronic excited states in deep variational Monte Carlo](https://mlsanity.com/paper/2203.09472/) (2022)
- [Sampling-free Inference for Ab-Initio Potential Energy Surface Networks](https://mlsanity.com/paper/2205.14962/) (2022)
- [A deep neural network for molecular wave functions in quasi-atomic minimal basis representation](https://mlsanity.com/paper/2005.06979/) (2020)

## Built on by (most cited first, 7 of 7)

- [Finetuning Foundation Models for Joint Analysis Optimization](https://mlsanity.com/paper/2401.13536/) (2024)
- [Variational Monte Carlo on a Budget -- Fine-tuning pre-trained Neural Wavefunctions](https://mlsanity.com/paper/2307.09337/) (2023)
- [Forward Laplacian: A New Computational Framework for Neural Network-based Variational Monte Carlo](https://mlsanity.com/paper/2307.08214/) (2023)
- [Sorting Out Quantum Monte Carlo](https://mlsanity.com/paper/2311.05598/) (2023)
- [Hamilton-Zero: A Neural Tensor-Network Foundation Model for Ground States of Arbitrary Quadratic Qubit Hamiltonians](https://mlsanity.com/paper/2608.11911/) (2026)
- [A Score-Based Model for Learning Neural Wavefunctions](https://mlsanity.com/paper/2305.16540/) (2023)
- [Is Variational Monte Carlo Robust? Sharp Moment Thresholds and Heavy-tailed Stochastic Optimization](https://mlsanity.com/paper/2606.26009/) (2026)

## Abstract (arXiv)

Deep neural networks have become a highly accurate and powerful wavefunction ansatz in combination with variational Monte Carlo methods for solving the electronic Schr"odinger equation. However, despite their success and favorable scaling, these methods are still computationally too costly for wide adoption. A significant obstacle is the requirement to optimize the wavefunction from scratch for each new system, thus requiring long optimization. In this work, we propose a novel neural network ansatz, which effectively maps uncorrelated, computationally cheap Hartree-Fock orbitals, to correlated, high-accuracy neural network orbitals. This ansatz is inherently capable of learning a single wavefunction across multiple compounds and geometries, as we demonstrate by successfully transferring a wavefunction model pre-trained on smaller fragments to larger compounds. Furthermore, we provide ample experimental evidence to support the idea that extensive pre-training of a such a generalized wavefunction model across different compounds and geometries could lead to a foundation wavefunction model. Such a model could yield high-accuracy ab-initio energies using only minimal computational effort for fine-tuning and evaluation of observables.