Duality-free Methods for Stochastic Composition Optimization

We consider the composition optimization with two expected-value functions in the form of frac1nsumnolimitsi = 1n Fi(frac1msumnolimitsj = 1m Gj(x))+R(x), which formulates many important problems in statistical learning and machine learning such as solving Bellman equations in reinforcement learning and nonlinear embedding. Full Gradient or classical stochastic gradient descent based optimization algorithms are unsuitable or computationally expensive to solve this problem due to the inner expectation frac1msumnolimitsj = 1m Gj(x). We propose a duality-free based stochastic composition method that combines variance reduction methods to address the stochastic composition problem. We apply SVRG and SAGA based methods to estimate the inner function, and duality-free method to estimate the outer function. We prove the linear convergence rate not only for the convex composition problem, but also for the case that the individual outer functions are non-convex while the objective function is strongly-convex. We also provide the results of experiments that show the effectiveness of our proposed methods.