Estimating the minimizer and the minimum value of a regression function under passive design

We propose a new method for estimating the minimizer boldsymbolx^* and the minimum value f^* of a smooth and strongly convex regression function f from the observations contaminated by random noise. Our estimator boldsymbolzn of the minimizer boldsymbolx^* is based on a version of the projected gradient descent with the gradient estimated by a regularized local polynomial algorithm. Next, we propose a two-stage procedure for estimation of the minimum value f^* of regression function f. At the first stage, we construct an accurate enough estimator of boldsymbolx^*, which can be, for example, boldsymbolzn. At the second stage, we estimate the function value at the point obtained in the first stage using a rate optimal nonparametric procedure. We derive non-asymptotic upper bounds for the quadratic risk and optimization error of boldsymbolzn, and for the risk of estimating f^*. We establish minimax lower bounds showing that, under certain choice of parameters, the proposed algorithms achieve the minimax optimal rates of convergence on the class of smooth and strongly convex functions.