mlsanityData to 14 Oct 2026

A Kaczmarz-inspired approach to accelerate the optimization of neural network wavefunctions

arXiv 2401.10190 ↗First posted 18 Jan 2024physics.comp-ph cs.LG physics.chem-phcited by 26

Not on the map. Why?

Abstract (arXiv)

Neural network wavefunctions optimized using the variational Monte Carlo method have been shown to produce highly accurate results for the electronic structure of atoms and small molecules, but the high cost of optimizing such wavefunctions prevents their application to larger systems. We propose the Subsampled Projected-Increment Natural Gradient Descent (SPRING) optimizer to reduce this bottleneck. SPRING combines ideas from the recently introduced minimum-step stochastic reconfiguration optimizer (MinSR) and the classical randomized Kaczmarz method for solving linear least-squares problems. We demonstrate that SPRING outperforms both MinSR and the popular Kronecker-Factored Approximate Curvature method (KFAC) across a number of small atoms and molecules, given that the learning rates of all methods are optimally tuned. For example, on the oxygen atom, SPRING attains chemical accuracy after forty thousand training iterations, whereas both MinSR and KFAC fail to do so even after one hundred thousand iterations.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 12 of its 43 references that are in the map. First posted on arXiv 18 Jan 2024.

Builds on

12 of 43 references.

References · 12

  1. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  2. 2019Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networkscited by 200
  3. 2019Deep neural network solution of the electronic Schr"odinger equationcited by 127
  4. 2019Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networkscited by 76
  5. 2022A Self-Attention Ansatz for Ab-initio Quantum Chemistrycited by 49
  6. 2019Efficient Subsampled Gauss-Newton and Natural Gradient Methods for Training Neural Networkscited by 33
  7. 2022Gradient Descent on Neurons and its Link to Approximate Second-Order Optimizationcited by 30
  8. 2022Gold-standard solutions to the Schr"odinger equation using deep learning: How much physics do we need?cited by 27
  9. 2022Discovering Quantum Phase Transitions with Fermionic Neural Networkscited by 26
  10. 2019Geometry of learning neural quantum statescited by 14
  11. 2021Explicitly antisymmetrized neural network layers for variational Monte Carlo simulationcited by 14
  12. 2021Rayleigh-Gauss-Newton optimization with enhanced sampling for variational Monte Carlocited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 9; 2026: 14.

26 papers.

Sort:

By area:

  1. 2025Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomizationcited by 13
  2. 2025Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learningcited by 12
  3. 2026Curvature-Aware Optimization for High-Accuracy Physics-Informed Neural Networkscited by 11
  4. 2024Neural Pfaffians: Solving Many Many-Electron Schr"odinger Equationscited by 9
  5. 2025Accurate Ab-initio Neural-network Solutions to Large-Scale Electronic Structure Problemscited by 8
  6. 2025Functional Neural Wavefunction Optimizationcited by 3
  7. 2025Worth Their Weight: Randomized and Regularized Block Kaczmarz Algorithms without Preprocessingcited by 2
  8. 2025A Sketch-and-Project Analysis of Subsampled Natural Gradient Algorithmscited by 2
  9. 2025Spin-Adapted Neural Network Wavefunctions in Real Spacecited by 2
  10. 2026Natural gradient descent with momentumcited by 2
  11. 2026Excited Pfaffians: Generalized Neural Wave Functions Across Structure and Statecited by 1
  12. 2026A Dirac-Frenkel-Onsager principle: Instantaneous residual minimization with gauge momentum for nonlinear parametrizations of PDE solutionscited by 1
14 more
  1. 2026Stochastic Reconfiguration as Statistical Filtering for Overparameterized Neural Quantum Statescited by 1
  2. 2024Ground state phases of the two-dimension electron gas with a unified variational approachcited by 0
  3. 2024Application of Langevin Dynamics to Advance the Quantum Natural Gradient Optimization Algorithmcited by 0
  4. 2025Neural-Quantum-States Impurity Solver for Quantum Embedding Problemscited by 0
  5. 2025Fermionic neural Gibbs statescited by 0
  6. 2026PROMA: Projected Microbatch Accumulation for Reference-Free Proximal Policy Updatescited by 0
  7. 2026Predicting magnetism with first-principles AIcited by 0
  8. 2026Randomized Advantage Transformation (RAT): Computing Natural Policy Gradients via Direct Backpropagationcited by 0
  9. 2026Dirac-Frenkel dynamics with inertia for nonlinearly parametrized solutions of evolution problemscited by 0
  10. 2026One More Time: Revisiting Neural Quantum States from a Reinforcement Learning Perspectivecited by 0
  11. 2026Enhanced Neural Quantum State via Annealed Gradient Descentcited by 0
  12. 2026Energy Manifold Natural Gradient Descent: Riemannian Optimization for Neural PDE Solverscited by 0
  13. 2026Backpropagated Output Momentum: Relocating Optimizer History from Parameters to Task Spacecited by 0
  14. 2026NucleoNet: A Nuclear-Aware Determinant Ansatz for Multi-Geometry Neural Variational Monte Carlocited by 0