# Gradient-estimator design overcomes trainability barriers in neural-network-based variational optimization

arXiv: https://arxiv.org/abs/2609.22342 · HTML: https://mlsanity.com/paper/2609.22342/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 48 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 12 of 12)

- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [Monte Carlo Gradient Estimation in Machine Learning](https://mlsanity.com/paper/1906.10652/) (2019)
- [Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networks](https://mlsanity.com/paper/1909.02487/) (2019)
- [Backpropagation through the Void: Optimizing control variates for black-box gradient estimation](https://mlsanity.com/paper/1711.00123/) (2017)
- [Understanding and correcting pathologies in the training of learned optimizers](https://mlsanity.com/paper/1810.10180/) (2018)
- [Deep autoregressive models for the efficient variational simulation of many-body quantum systems](https://mlsanity.com/paper/1902.04057/) (2019)
- [PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos](https://mlsanity.com/paper/1902.01240/) (2019)
- [A Self-Attention Ansatz for Ab-initio Quantum Chemistry](https://mlsanity.com/paper/2211.13672/) (2022)
- [Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives](https://mlsanity.com/paper/1810.04152/) (2018)
- [Autoregressive neural-network wavefunctions for ab initio quantum chemistry](https://mlsanity.com/paper/2109.12606/) (2021)
- [Neural network wave functions and the sign problem](https://mlsanity.com/paper/2002.04613/) (2020)

## Abstract (arXiv)

Neural networks provide expressive representations for scientific computing. However, even sufficiently expressive networks can suffer training failure in weak-gradient regimes, limiting their practical use in quantum many-body physics and ab initio quantum chemistry. Here we derive an unbiased direct gradient estimator and introduce the adaptive minimum-variance phase (AMVP) estimator for neural-network variational optimization. By improving the signal-to-noise ratio of weak gradients, these methods enable reliable scientific calculations where training previously failed, while substantially reducing computational cost. The framework enables compact networks to outperform larger and fine-tuned default standard-estimator models with over an order of magnitude less GPU time on correlated flux models, and ultimately exceed the density matrix renormalization group (DMRG) accuracy. It further achieves chemical accuracy in N2 bond breaking and, for the first time, in heavy-element I2 with explicit spin-orbit coupling. These results demonstrate that gradient-estimator design expands the capabilities of neural-network variational methods for accurate scientific computing.