# Higher-order Quasi-Monte Carlo Training of Deep Neural Networks

arXiv: https://arxiv.org/abs/2009.02713 · HTML: https://mlsanity.com/paper/2009.02713/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Sep 2020.

## Built on by (most cited first, 12 of 12)

- [On the approximation of functions by tanh neural networks](https://mlsanity.com/paper/2104.08938/) (2021)
- [Message-Passing Monte Carlo: Generating low-discrepancy point sets via Graph Neural Networks](https://mlsanity.com/paper/2405.15059/) (2024)
- [Langevin Quasi-Monte Carlo](https://mlsanity.com/paper/2309.12664/) (2023)
- [Lattice-based Deep Neural Networks: Regularity and Tailored Regularization](https://mlsanity.com/paper/2603.02809/) (2026)
- [Convergence analysis of a quasi-Monte Carlo-based deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/2210.16196/) (2022)
- [Convergence analysis of OT-Flow for sample generation](https://mlsanity.com/paper/2403.16208/) (2024)
- [Using Low-Discrepancy Points for Data Compression in Machine Learning: An Experimental Comparison](https://mlsanity.com/paper/2407.07450/) (2024)
- [Neural Low-Discrepancy Sequences](https://mlsanity.com/paper/2510.03745/) (2025)
- [Number Theoretic Accelerated Learning of Physics-Informed Neural Networks](https://mlsanity.com/paper/2307.13869/) (2023)
- [A Bayesian Approach to Low-Discrepancy Subset Selection](https://mlsanity.com/paper/2602.14607/) (2026)
- [Fourier Neural Operators with rank-1 lattice points and hyperbolic cross](https://mlsanity.com/paper/2606.08871/) (2026)
- [High Probability Derivative Bounds for Random tanh Neural Networks on a Hypercube](https://mlsanity.com/paper/2608.26526/) (2026)

## Abstract (arXiv)

We present a novel algorithmic approach and an error analysis leveraging Quasi-Monte Carlo points for training deep neural network (DNN) surrogates of Data-to-Observable (DtO) maps in engineering design. Our analysis reveals higher-order consistent, deterministic choices of training points in the input data space for deep and shallow Neural Networks with holomorphic activation functions such as tanh. These novel training points are proved to facilitate higher-order decay (in terms of the number of training samples) of the underlying generalization error, with consistency error bounds that are free from the curse of dimensionality in the input data space, provided that DNN weights in hidden layers satisfy certain summability conditions. We present numerical experiments for DtO maps from elliptic and parabolic PDEs with uncertain inputs that confirm the theoretical analysis.