Stochastic Trace Optimization of Parameter Dependent Matrices Based on Statistical Learning Theory

We consider matrices boldsymbolA(boldsymbolθ)∈ℝm× m that depend, possibly nonlinearly, on a parameter boldsymbolθ from a compact parameter space Theta. We present a Monte Carlo estimator for minimizing trace(boldsymbolA(boldsymbolθ)) over all boldsymbolθ∈Theta, and determine the sampling amount so that the backward error of the estimator is bounded with high probability. We derive two types of bounds, based on epsilon nets and on generic chaining. Both types predict a small sampling amount for matrices boldsymbolA(boldsymbolθ) with small offdiagonal mass, and parameter spaces Theta of small ``size.'' Dependence on the matrix dimension~m is only weak or not explicit. The bounds based on epsilon nets are easier to evaluate and come with fully specified constants. In contrast, the bounds based on chaining depend on the Talagrand functionals which are difficult to evaluate, except in very special cases. Comparisons between the two types of bounds are difficult, although the literature suggests that chaining bounds can be superior.