Sufficient conditions for CQ.

5.1. QUALITATIVE THEORY OF NONLINEAR PROGRAMMING 55 f i θy + 1 − θz ≤ θf i y + 1 − θf i z ≤ 0 , 1 ≤ i ≤ m, so that θy + 1 − θz ∈ Ω, hence Ω is convex. ii Let x ∈ Ω be arbitrary. Since f is concave, by Exercise 6 we have f x ≤ f x ∗ + f 0x x ∗ x − x ∗ , so that by 5.8 f x ≤ f x ∗ + X i∈Ix ∗ λ ∗ i f ix x ∗ x − x ∗ . 5.13 Next, f i is convex so that again by Exercise 6, f i x ≥ f i x ∗ + f ix x ∗ x − x ∗ ; but f i x ≤ 0, and f i x ∗ = 0 for i ∈ Ix ∗ , so that f ix x ∗ x − x ∗ ≤ 0 for i ∈ Ix ∗ . 5.14 Combining 5.14 with the fact that λ ∗ i ≥ 0, we conclude from 5.13 that f x ≤ f x ∗ , so that x ∗ is optimal. ♦ Exercise 7: Under the hypothesis of Theorem 4, show that the subset Ω ∗ of Ω, consisting of all the optimal solutions of 5.1, is a convex set. Exercise 8: A function h : X → R defined on a convex set X ⊂ R n is said to be strictly convex if hθy + 1 − θz θhy + 1 − θhz whenever 0 θ 1 and y, z are in X with y 6= z. h is said to be strictly concave if −h is strictly convex. Under the hypothesis of Theorem 4, show that an optimal solution to 5.1 is unique if it exists if either f is strictly concave or if the f i , 1 ≤ i ≤ m, are strictly convex. Hint: Show that in 5.13 we have strict inequality if x 6= x ∗ .

5.1.3 Sufficient conditions for CQ.

As stated, it is usually impractical to verify if CQ is satisfied for a particular problem. In this subsection we give two conditions which guarantee CQ. These conditions can often be verified in practice. Recall that a function g : R n → R is said to be affine if gx ≡ α + b ′ x for some fixed α ∈ R and b ∈ R n . We adopt the formulation 5.1 so that Ω = {x ∈ R n |f i x ≤ 0 , 1 ≤ i ≤ m} . Lemma 3: Suppose x ∗ ∈ Ω and suppose there exists h ∗ ∈ R n such that for each i ∈ Ix ∗ , either f ix x ∗ h ∗ 0, or f ix x ∗ h ∗ = 0 and f i is affine. Then CQ is satisfied at x ∗ . Proof: Let h ∈ R n be such that f ix x ∗ h ≤ 0 for i ∈ Ix ∗ . Let δ 0. We will first show that h + δh ∗ ∈ CΩ, x ∗ . To this end let ε k 0, k = 1, 2, . . . , be a sequence converging to 0 and set x k = x ∗ + ε k h + δh ∗ . Clearly x k converges to x ∗ , and 1ε k x k − x ∗ converges to h + δh ∗ . Also for i ∈ Ix ∗ , if f ix x ∗ h 0, then f i x k = f i x ∗ + ε k f ix x ∗ h + δh ∗ + oε k |h + δh ∗ | ≤ δε k f ix x ∗ h ∗ + oε k |h + δh ∗ | 0 for sufficiently large k , whereas for i ∈ Ix ∗ , if f i is affine, then 56 CHAPTER 5. NONLINEAR PROGRAMMING f i x k = f i x ∗ + ε k f ix x ∗ h + δh ∗ ≤ 0 for all k . Finally, for i 6∈ Ix ∗ we have f i x ∗ 0, so that f i x k 0 for sufficiently large k. Thus we have also shown that x k ∈ Ω for sufficiently large k, and so by definition h + δh ∗ ∈ CΩ, x ∗ . Since δ 0 can be arbitrarily small, and since CΩ, x ∗ is a closed set by Exercise 2, it follows that h ∈ CΩ, x ∗ . ♦ Exercise 9: Suppose x ∗ ∈ Ω and suppose there exists ˆx ∈ R n such that for each i ∈ Ix ∗ , either f i x ∗ 0 and f i is convex, or f i ˆ x ≤ 0 and f i is affine. Then CQ is satisfied at x ∗ . Hint: Show that h ∗ = ˆ x − x ∗ satisfies the hypothesis of Lemma 3. Lemma 4: Suppose x ∗ ∈ Ω and suppose there exists h ∗ ∈ R n such that f ix x ∗ h ∗ ≤ 0 for i ∈ Ix ∗ , and {f ix x ∗ |i ∈ Ix ∗ , f ix x ∗ h ∗ = 0} is a linearly independent set. Then CQ is satisfied at x ∗ . Proof: Let h ∈ R n be such that f ix x ∗ h ≤ 0 for all i ∈ Ix ∗ . Let δ 0. We will show that h + δh ∗ ∈ CΩ, x ∗ . Let J δ = {i|i ∈ Ix ∗ , f ix x ∗ h + δh ∗ = 0}, consist of p elements. Clearly J δ ⊂ J = {i|i ∈ Ix ∗ , f i xx ∗ h ∗ = 0}, so that {f ix x ∗ , u ∗ |i ∈ J δ } is linearly independent. By the Implicit Function Theorem, there exist ρ 0, an open set V ⊂ R n containing x ∗ = w ∗ , u ∗ , and a differentiable function g : U → R p , where U = {u ∈ R n−p ||u − u ∗ | ρ}, such that f i w, u = 0, i ∈ J δ , and w, u ∈ V iff u ∈ U, and w = gu . Next we partition h, h ∗ as h = ξ, η, h ∗ = ξ ∗ , η ∗ corresponding to the partition of x = w, u. Let ε k 0, k = 1, 2 . . . , be any sequence converging to 0, and set u k = u ∗ + ε k η + δη ∗ , w k = gu k , and finally x k = s k , u k . We note that u k converges to u ∗ , so w k = gu k converges to w ∗ = gu ∗ . Thus, x k converges to x ∗ . Now 1ε k x k − x ∗ = 1ε k w k − w ∗ , u k − u ∗ = 1ε k gu k − gu ∗ , ε k η + δη ∗ . Since g is differentiable, it follows that 1ε k x k − x ∗ converges to g u u ∗ η + δη ∗ , η + δη ∗ . But for i ∈ J δ we have 0 = f ix x ∗ h + δh ∗ = f iw x ∗ ξ + δξ ∗ + f iu x ∗ η + δη ∗ . 5.15 Also, for i ∈ J δ , 0 = f i gu, u for u ∈ U so that 0 = f iw x ∗ g u u ∗ + f iu x ∗ , and hence 0 = f iw x ∗ g u u ∗ η + δη ∗ + f iu x ∗ η + δη ∗ . 5.16 If we compare 5.15 and 5.16 and recall that {f iw x ∗ |i ∈ J δ } is a basis in R p we can conclude that ξ + δξ ∗ = g u u ∗ η + δη ∗ so that 1ε k x k − x ∗ converges to h + hδh ∗ . It remains to show that x k ∈ Ω for sufficiently large k. First of all, for i ∈ J δ , f i x k = f i gu k , u k = 0, whereas for i 6∈ J δ , i ∈ Ix ∗ , f i x k = f i x ∗ + f ix x ∗ x k − x ∗ + o|x k − x ∗ | f i x ∗ + ε k f ix x ∗ h + δh ∗ + oε k + o|x k − x ∗ |, 5.2. DUALITY THEORY 57 and since f i x ∗ = 0 whereas f ix x ∗ h + δh ∗ 0, we can conclude that f i x k 0 for suffi- ciently large k. Thus, x k ∈ Ω for sufficiently large k. Hence, h + δh ∗ ∈ CΩ, x ∗ . To finish the proof we note that δ 0 can be made arbitrarily small, and CΩ, x ∗ is closed by Exercise 2, so that h ∈ CΩ, x ∗ . ♦ The next lemma applies to the formulation 5.9. Its proof is left as an exercise since it is very similar to the proof of Lemma 4. Lemma 5: Suppose x ∗ is feasible for 5.9 and suppose there exists h ∗ ∈ R n such that the set {f ix x ∗ |i ∈ Ix ∗ , f ix x ∗ h ∗ = 0} S{r jx x ∗ |j = 1, . . . , k} is linearly independent, and f ix x ∗ h ∗ ≤ 0 for i ∈ Ix ∗ , r jx x ∗ h ∗ = 0 for 1 ≤ j ≤ k. Then CQ is satisfied at x ∗ . Exercise 10: Prove Lemma 5

5.2 Duality Theory