Universal nonmonotone line search method for nonconvex multiobjective optimization problems with convex constraints

In this work we propose a general nonmonotone line-search method for nonconvex multi-objective optimization problems with convex constraints. At the kth iteration, the degree of nonmonotonicity is controlled by a vector nuk with nonnegative components. Different choices for nuk lead to different nonmonotone step-size rules. Assuming that the sequence leftnukrightk≥ 0 is summable, and that the ith objective function has H"older continuous gradient with smoothness parameter θi ∈(0,1], we show that the proposed method takes no more than Oleft(ε^-left(1+frac1θminright)right) iterations to find a ε-approximate Pareto critical point for a problem with m objectives and θmin= mini=1,dots, m θi. In particular, this complexity bound applies to the methods proposed by Drummond and Iusem (Comput. Optim. Appl. 28: 5--29, 2004), by Fazzio and Schuverdt (Optim. Lett. 13: 1365--1379, 2019), and by Mita, Fukuda and Yamashita (J. Glob. Optim. 75: 63--90, 2019). The generality of our approach also allows the development of new methods for multiobjective optimization. As an example, we propose a new nonmonotone step-size rule inspired by the Metropolis criterion. Preliminary numerical results illustrate the benefit of nonmonotone line searches and suggest that our new rule is particularly suitable for multiobjective problems in which at least one of the objectives has many non-global local minimizers.