PARALLEL SYNCRHRONOUS ALGORITHM FOR NONLINEAR FIXED POINT PROBLEMS

We give in this paper a convergence result concerning parallel synchronous algorithm for nonlinear fixed point problems with respect to the euclidean norm in R. We then apply this result to some problems related to convex analysis like minimization of functionals, calculus of saddle point, convex programming...


Introduction.
This study is motivated by the paper of Bahi [3] where he has given a convergence result concerning parallel synchronous algorithm for linear fixed point problems using nonexpansive linear mappings with respect to a weighted maximum norm.Our goal is to extend this result to a nonlinear fixed point problems, with respect to the euclidean norm, where F : R n → R n is a nonlinear operator.Section 2 is devoted to a brief description of asynchronous parallel algorithm.In section 3 we prove the main result concerning the convergence of the general algorithm in the synchronous case to a fixed point of a nonlinear operator from R n to R n .To prove this result, we have assumed that F is a "contraction" in a senses more general than the nonexpansitiveness notion.A particular case of this algorithm (Algorithm of Jacobi) is applied in section 4 to the operator F = (I + T ) −1 which is called the proximal mapping associated with the maximal monotone operator T (see Rockafellar [9]).

Preliminaries on asynchronous algorithms.
Asynchronous algorithms are used in the parallel treatment of problems taking in consideration the interaction of several processors.Write R n as the product x ∈ R n considered in this study are splitted in the form x = (x 1 , ..., x α ) where x i ∈ R n i .Let R n i be equipped with the inner product h., .ii and the associated norm k.k i = h., .i1/2 i .R n will be equipped with the inner product hx, yi = α P i=1 hx i , y i i i where x, y ∈ R n and the norm kxk Define : J = {J(p)} p∈N a sequence of non empty sub sets of {1, ..., α} and Consider an operator F = (F 1 , ..., F α ) R n → R n and define the asynchronous algorithm associated with F by (see Bahi and al. [1], El Tarazi [4]), It will be denoted by (F, x 0 , J, S).This algorithm describes the behaviour of iterative process executed asynchronously on a parallel computer with α processors.At each iteration p + 1, the i th processor computes x p+1 i by using (2.1) (Bahi [2]).J(p) is the subset of the indexes of the components updated at the p th step.p−s i (p) is the delay due to the i th processor when it computes the i th block at the p th iteration.If we take s i (p) = p ∀i ∈ {1, ..., α}, then (2.1) describes synchronous algorithm (without delay).During each iteration, every processor executes a number of computations that depend on the results of the computations of other processors in the previous iteration.Within an iteration, each processor does not interact with other processors, all interactions takes place at the end of iterations (Bahi[3]).
Remark 1.The hypothesis (h 0 ) means that at times (iterations) the processors are synchronized and all the components are updated.This subsequence can be chosen by the programmer (Bahi [2]).

Convergence of the general algorithm.
We establish in this section the convergence of the general parallel synchronous algorithm to a fixed point of a nonlinear operator F : R n → R n with respect to the euclidean norm defined in section 2. First, we define a notion which will be used in the later and which generalize the nonexpansiveness 3 notion of an operator F .
For example, the map any parallel synchronous4 algorithm defined by (2.1) associated with the operator F converges to a fixed point x * of F .

Proof. [i]
We prove first that the sequence {x p } p∈N is bounded.
Inductivelly it follows that kx p − uk ≤ °°x 0 − u °°, so that the sequence {x p } p∈N is bounded.
[ii] The sequence {x p k } k∈N (where {p k } k∈N is defined by (h 0 )) contains a subsequence noted also {x p k } k∈N which is convergent in R n to an x * .We show that x * is a fixed point of F .For it, we consider the sequence {y p = x p − F (x p )} p∈N and prove that lim k→∞ y p k = 0.
However, in (i) we have shown in particular that the sequence {kx p − uk} p∈N is decreasing (positive), it's therefore convergent, so that is x * is a fixed point of F .
[iii] We prove as in (i) that the sequence {kx p − x * k} p∈N is decreasing (positive) then it is convergent, so Remark 2. We have used the hypothesis (h 3 ) to prove that the sequence {x p } p∈N is bounded.In the case of the parallel algorithm of Jacobi where J(p) = {1, ..., α} ∀p ∈ N, we don't need this hypothesis, since in this case x p+1 = F (x p ) ∀p ∈ N, and use (h 2 ) to obtain hence the corollary, Corollary 3.Under the hypotheses (h 1 ), (h 2 ) and (h 0 0 ) ∀p ∈ N, J(p) = {1, ..., α} The parallel Jacobi algorithm defined by converges in R n to an x * fixed point of F .

Solutions of maximal monotone operators.
In this section, we apply the parallel Jacobi algorithm to the proximal mapping F = (I + T ) −1 associated with the maximal monotone operator T .We give first a general result concerning the maximal monotone operators.Such operators have been studied extensively because of their role in convex analysis (minimization of functionals, min-max problems, convex programming, ...) and certain partial differential equations (Rockafellar[9]).
Let T be a multivalued maximal monotone operator defined from R n to R n .A fundamental problem is to determine an x * in R n satisfying 0 ∈ T x * which will be called a solution of the operator T .Theorem 4. Let T be a multivalued maximal monotone operator such that T −1 0 6 =.Then every parallel Jacobi algorithm associated with the single-valued mapping F = (I + T ) −1 converges in R n to an x * solution of the problem 0 ∈ T x.
Thus, the solutions of T are the fixed points of F , so the condition T −1 0 6 = implies the existence of a fixed point u of R n .It remains to show that F verifies the condition (h 2 ) and apply Corollary 3. Consider and put y i = F x i then x i ∈ y i + T y i or x i − y i ∈ T y i .As T is monotone we have (x 1 − y 1 ) − (x 2 − y 2 ), y 1 − y 2 ® ≥ 0 and therefore 2. Minimization of functional.
Corollary 5. Let f : R n → R ∪ {∞} be a lower semicontinuous convex function which is proper (i.e not identically +∞).Suppose that the minimization problem min R n f (x) has a solution.Then any parallel Jacobi algorithm associated with the single-valued mapping F = (I + ∂f ) −1 converges to a minimizer of f in R n .

Proof.
Since in this case the subdifferential ∂f is maximal monotone.Moreover the minimizers of f are the solutions of ∂f .We then apply Theorem 4 to ∂f.If (x * , y * ) is a saddle point of the Lagrangian L then x * is an optimal solution of the primal problem (P ) and y * is an optimal solution of the dual problem (D).
Let ∂L(x, y) the subdifferential of L at (x, y) ∈ R n × p , be defined as the set of vectors (u, v) ∈ R n × p satisfying (see Luque [5] and Rockafellar [6]).
Corollary 7. Suppose that the convex programming (P ) defined by (4.2) has a solution.Then any parallel Jacobi algorithm associated with the single-valued mapping F = (I + T L ) −1 from R n × p to R n × p converges to a saddle point (x * , y * ) of L, and so x * is a solution of the primal (P ) and y * a solution of the dual (D).

Variational inequality.
A simple formulation of the variational inequality problem is to find an x * ∈ R n satisfying hAx * , x − x * i ≥ 0 ∀x ∈ R n (4.4) where A : R n → R n is a single-valued monotone and maximal operator 5 .Which is equivalent to find an x * ∈ R n such that 0 ∈ Ax * + N (x * ) where N (x) is the normal cone to R n at x defined by (see Rockafellar [6,9]), N (x) = {y ∈ R n : hy, x − zi ≥ 0 ∀z ∈ R n } Rockafellar [9] has considered the multifunction T defined in R n by T x = Ax + N (x) (4.5) and shown in [8] that T is maximal monotone.The relation 0 ∈ T x * is so that reduced to −Ax * ∈ N (x * ) or h−Ax * , x * − zi ≥ 0 ∀z ∈ R n which is the variational inequality (4.4).Therefore the solutions of the operator T (defined by (4.5)) are exactly the solutions of the variational inequality (4.4).By using Theorem 4 we can write Corollary 8. Let A : R n → R n be a single-valued monotone and hemicontinuous operator such that the problem (4.4) has a solution, then any parallel Jacobi algorithm associated with the single-valued mapping F = (I + T ) −1 (where T is defined by (4.5)) converges to x * solution of the problem (4.4).

4. 3 .
Saddle point.In this paragraph, we apply Theorem 4 to calculate a saddle point of functional L :R n × p → [−∞, +∞].Recall that a saddle point of L is an element (x * , y * ) of R n × p satisfying L(x * , y) ≤ L(x * , y * ) ≤ L(x, y * ), ∀(x, y) ∈ R n × p which is equivalent to L(x * , y * ) = inf x∈R n L(x, y * ) = sup y∈Rp L(x * , y)Suppose that L(x, y) is convex lower semicontinuous in x ∈ R n and concave upper semicontinuous in y ∈ Rp.