5.1.
QUALITATIVE THEORY OF NONLINEAR PROGRAMMING 53
CΩ, x
∗
6= {h|f
ix
x
∗
h ≤ 0 , i = 1, 2, }. Note that
{f
1x
x
∗
, f
2x
x
∗
} is not a linearly independent set; see Lemma 4 below.
5.1.2 Kuhn-Tucker Theorem.
Definition: Let
x
∗
∈ Ω. We say that the constraint qualification CQ is satisfied at x
∗
if CΩ, x = {h|f
ix
x
∗
h ≤ 0 for all i ∈ Ix
∗
}, and we say that CQ is satisfied if CQ is satisfied at all
x ∈ Ω. Note that by Lemma 2 CΩ, x is always a subset of the right-hand side.
Compare the next result with Exercise 2 of 4.2. Theorem 1:
Kuhn and Tucker [1951] Let x
∗
be an optimum solution of 5.1, and suppose that CQ is satisfied at
x
∗
. Then there exist λ
∗ i
≥ 0, for i ∈ Ix
∗
, such that f
0x
x
∗
= X
i∈Ix
∗
λ
∗ i
f
ix
x
∗
5.8 Proof:
By Lemma 1 and the definition of CQ it follows that f
0x
x
∗
h ≤ 0 whenever f
ix
x
∗
h ≤ 0 for all
i ∈ Ix
∗
. By the Farkas’ Lemma of 4.2.1 it follows that there exist λ
∗ i
≥ 0 for i ∈ Ix
∗
such that 5.8 holds. ♦
In the original formulation of the decision problem we often have equality constraints of the form r
j
x = 0, which get replaced by r
j
x ≤ 0, −r
j
x ≤ 0 to give the form 5.1. It is convenient in application to separate the equality constraints from the rest. Theorem 1 can then be expressed as
Theorem 2.
Theorem 2: Consider the problem 5.9.
Maximize f
x subject to
f
i
x ≤ 0 , i = 1, . . . , m, r
j
x = 0 , j = 1, . . . , k . 5.9
Let x
∗
be an optimum decision and suppose that CQ is satisfied at x
∗
. Then there exist λ
∗ i
≥ 0, i = 1, . . . , m, and µ
∗ j
, j = 1, . . . , k such that f
0x
x
∗
=
m
X
i=1
λ
∗ i
f
ix
x
∗
+
k
X
j=1
µ
∗ j
r
jx
x
∗
, 5.10
and λ
∗ i
= 0 whenever f
i
x
∗
0 . 5.11
Exercise 4: Prove Theorem 2.
54
CHAPTER 5. NONLINEAR PROGRAMMING
An alternative form of Theorem 1 will prove useful for computational purposes see Section 4. Theorem 3:
Consider 5.9, and suppose that CQ is satisfied at an optimal solution x
∗
. Define ψ : R
n
→ R by ψh = max {−f
0x
x
∗
h, f
1
x
∗
+ f
1x
x
∗
h, . . . , f
m
x
∗
+ f
mx
x
∗
h} , and consider the decision problem
Minimize ψh
subject to −ψh − f
0x
x
∗
h ≤ 0, −ψh + f
i
x
∗
+ f
ix
x
∗
h ≤ 0 , 1 ≤ i ≤ m −1 ≤ h
i
≤ 1 , i = 1, . . . , n . 5.12
Then h = 0 is an optimal solution of 5.12.
Exercise 5: Prove Theorem 3. Note that by Exercise 1 of 4.5, 5.12 can be transformed into a
LP. Remark:
For problem 5.9 define the Lagrangian function L:
x
1
, . . . , x
n
; λ
1
, . . . , λ
m
; µ
1
, . . . , µ
k
7→ f x −
m
X
i=1
λ
i
f
i
x −
k
X
j=1
µ
j
r
j
x. Then Theorem 2 is equivalent to the following statement: if
CQ is satisfied and x
∗
is optimal, then there exist
λ
∗
≥ 0 and µ
∗
such that L
x
x
∗
, λ
∗
, µ
∗
= 0 and Lx
∗
, λ
∗
, µ
∗
≤ Lx
∗
, λ, µ for all λ ≥ 0, µ.
There is a very important special case when the necessary conditions of Theorem 1 are also sufficient. But first we need some elementary properties of convex functions which are stated as an
exercise. Some additional properties which we will use later are also collected here. Recall the definition of convex and concave functions in 4.2.3.
Exercise 6:
Let X ⊂ R
n
be convex. Let h : X → R be a differentiable function. Then
i h is convex iff hy ≥ hx + h
x
xy − x for all x, y, in X, ii
h is concave iff hy ≤ hx + h
x
xy − x for all x, y in X, iii
h is concave and convex iff h is affine, i.e. hx ≡ α + b
′
x for some fixed
α ∈ R, b ∈ R
n
. Suppose that
h is twice differentiable. Then iv
h is convex iff h
xx
x is positive semidefinite for all x in X, v
h is concave iff h
xx
x is negative semidefinite for all x in X, vi
h is convex and concave iff h
xx
x ≡ 0. Theorem 4:
Sufficient condition In 5.1 suppose that f
is concave and f
i
is convex for i = 1, . . . , m. Then
i Ω is a convex subset of R
n
, and ii if there exist
x
∗
∈ Ω, λ
∗ i
≥ 0, i ∈ Ix
∗
, satisfying 5.8, then x
∗
is an optimal solution of 5.1.
Proof: i Let
y, z be in Ω so that f
i
y ≤ 0, f
i
z ≤ 0 for i = 1, . . . , m. Let 0 ≤ θ ≤ 1. Since f
i
is convex we have
5.1.
QUALITATIVE THEORY OF NONLINEAR PROGRAMMING 55
f
i
θy + 1 − θz ≤ θf
i
y + 1 − θf
i
z ≤ 0 , 1 ≤ i ≤ m, so that
θy + 1 − θz ∈ Ω, hence Ω is convex. ii Let
x ∈ Ω be arbitrary. Since f is concave, by Exercise 6 we have
f x ≤ f
x
∗
+ f
0x
x
∗
x − x
∗
, so that by 5.8
f x ≤ f
x
∗
+ X
i∈Ix
∗
λ
∗ i
f
ix
x
∗
x − x
∗
. 5.13
Next, f
i
is convex so that again by Exercise 6, f
i
x ≥ f
i
x
∗
+ f
ix
x
∗
x − x
∗
; but
f
i
x ≤ 0, and f
i
x
∗
= 0 for i ∈ Ix
∗
, so that f
ix
x
∗
x − x
∗
≤ 0 for i ∈ Ix
∗
. 5.14
Combining 5.14 with the fact that λ
∗ i
≥ 0, we conclude from 5.13 that f x ≤ f
x
∗
, so that x
∗
is optimal. ♦
Exercise 7:
Under the hypothesis of Theorem 4, show that the subset Ω
∗
of Ω, consisting of all the
optimal solutions of 5.1, is a convex set. Exercise 8:
A function h : X → R defined on a convex set X ⊂ R
n
is said to be strictly convex if hθy + 1 − θz θhy + 1 − θhz whenever 0 θ 1 and y, z are in X with y 6= z. h is
said to be strictly concave if −h is strictly convex. Under the hypothesis of Theorem 4, show that
an optimal solution to 5.1 is unique if it exists if either f
is strictly concave or if the f
i
, 1 ≤ i ≤ m, are strictly convex. Hint: Show that in 5.13 we have strict inequality if x 6= x
∗
.
5.1.3 Sufficient conditions for CQ.
