Online Convex Optimization with Time-Varying Constraints

This paper considers online convex optimization with time-varying constraint functions. Specifically, we have a sequence of convex objective functions ft(x)t=0∞ and convex constraint functions gt,i(x)t=0∞ for i ∈ 1, ..., k. The functions are gradually revealed over time. For a given ε>0, the goal is to choose points xt every step t, without knowing the ft and gt,i functions on that step, to achieve a time average at most ε worse than the best fixed-decision that could be chosen with hindsight, subject to the time average of the constraint functions being nonpositive. It is known that this goal is generally impossible. This paper develops an online algorithm that solves the problem with O(1/ε2) convergence time in the special case when all constraint functions are nonpositive over a common subset of ℝn. Similar performance is shown in an expected sense when the common subset assumption is removed but the constraint functions are assumed to vary according to a random process that is independent and identically distributed (i.i.d.) over time slots t ∈ 0, 1, 2, ldots. Finally, in the special case when both the constraint and objective functions are i.i.d. over time slots t, the algorithm is shown to come within ε of optimality with respect to the best (possibly time-varying) causal policy that knows the full probability distribution.