mlsanityData to 14 Oct 2026

Convergence of variational Monte Carlo simulation and scale-invariant pre-training

arXiv 2303.11602 ↗First posted 21 Mar 2023cs.LG physics.comp-phcited by 2

Not on the map. Why?

Abstract (arXiv)

We provide theoretical convergence bounds for the variational Monte Carlo (VMC) method as applied to optimize neural network wave functions for the electronic structure problem. We study both the energy minimization phase and the supervised pre-training phase that is commonly used prior to energy minimization. For the energy minimization phase, the standard algorithm is scale-invariant by design, and we provide a proof of convergence for this algorithm without modifications. The pre-training stage typically does not feature such scale-invariance. We propose using a scale-invariant loss for the pretraining phase and demonstrate empirically that it leads to faster pre-training.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 23 of its 52 references that are in the map. First posted on arXiv 21 Mar 2023.

Builds on

23 of 52 references.

References · 23

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  2. 2018Group Normalizationcited by 1,524
  3. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 543
  4. 2014SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectivescited by 402
  5. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
  6. 2019Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networkscited by 200
  7. 2013Minimizing Finite Sums with the Stochastic Average Gradientcited by 164
  8. 2023Handbook of Convergence Theorems for (Stochastic) Gradient Methodscited by 163
  9. 2020Better Theory for SGD in the Nonconvex Worldcited by 158
  10. 2019Deep neural network solution of the electronic Schr"odinger equationcited by 127
  11. 2017L2 Regularization versus Batch and Weight Normalizationcited by 124
  12. 2018Theoretical Analysis of Auto Rate-Tuning by Batch Normalizationcited by 65
11 more
  1. 2016Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifoldscited by 64
  2. 2022A Self-Attention Ansatz for Ab-initio Quantum Chemistrycited by 49
  3. 2018Averaging Stochastic Gradient Descent on Riemannian Manifoldscited by 39
  4. 2022Robust Training of Neural Networks Using Scale Invariant Architecturescited by 29
  5. 2020Better, Faster Fermionic Neural Networkscited by 28
  6. 2018G-SGD: Optimizing ReLU Neural Networks in its Positively Scale-Invariant Spacecited by 27
  7. 2022Gold-standard solutions to the Schr"odinger equation using deep learning: How much physics do we need?cited by 27
  8. 2022Discovering Quantum Phase Transitions with Fermionic Neural Networkscited by 26
  9. 2021Autoregressive neural-network wavefunctions for ab initio quantum chemistrycited by 21
  10. 2022Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimescited by 19
  11. 2020Phases of two-dimensional spinless lattice fermions with first-quantized deep neural-network quantum statescited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Accurate Ab-initio Neural-network Solutions to Large-Scale Electronic Structure Problemscited by 8
  2. 2026Is Variational Monte Carlo Robust? Sharp Moment Thresholds and Heavy-tailed Stochastic Optimizationcited by 0