Differential estimates for fast first-order multilevel nonconvex optimisation

With a view on bilevel and PDE-constrained optimisation, we develop iterative estimates widetildeF'(xk) of F'(xk) for composite functions F :=J circ S, where S is the solution mapping of the inner optimisation problem or PDE. The idea is to form a single-loop method by interweaving updates of the iterate xk by an outer optimisation method, with updates of the estimate by single steps of standard optimisation methods and linear system solvers. When the inner methods satisfy simple tracking inequalities, the differential estimates can almost directly be employed in standard convergence proofs for general forward-backward type methods. We adapt those proofs to a general inexact setting in normed spaces, that, besides our differential estimates, also covers mismatched adjoints and unreachable optimality conditions in measure spaces. As a side product of these efforts, we provide improved convergence results for nonconvex Primal-Dual Proximal Splitting (PDPS). We numerically evaluate our methods on Electrical Impedance Tomography (EIT) and minimal surface control.