Convergence of the operator extrapolation method

Abstract

One of the popular areas of the modern applied nonlinear analysis is the study of variational inequalities and the development of methods for approximating their solutions. Many important problems of the research of operations, optimal control theory, and mathematical physics can be written in the form of variational inequalities. Non-smooth optimization problems can be solved effectively, if they are reformulated as saddle problems, and modern approximate algorithms for solving the variational inequalities are applied to the obtained saddle problems. With the advent of generating adversarial neural networks (GANs), the strong interest in the use and investigation of iterative algorithms for solving the variational inequalities arose in the ML-community. This paper is devoted to the study of two new approximate algorithms with the Bregman projection for solving the variational inequalities in a Hilbert space. The first algorithm, which we call the operator extrapolation algorithm, is obtained by replacing the Euclidean metric in the Malitsky–Tam method with the Bregman divergence. An attractive feature of the algorithm is only one computation at the iterative step of the Bregman projection onto the feasible set. The second algorithm is an adaptive version of the first, where the used rule for updating the step size does not require knowledge of Lipschitz constants and the calculation of operator values at additional points. For variational inequalities with pseudo-monotone, Lipschitz-continuous, and sequentially weakly continuous operators acting in a Hilbert space, some weak convergence theorems are proved.