5.4.
COMPUTATIONAL METHOD 71
subject to Ax + I
m
y + z
= b −P x
− A
′
λ + I
n
µ + ξ = −c
x ≥ 0, y ≥ 0, λ ≥ 0, µ ≥ 0, z ≥ 0, ξ ≥ 0, 5.44
starting with a basic feasible solution z = b, ξ = −c. We have assumed, without loss of generality,
that b ≥ 0 and −c ≥ 0. If 5.41 and 5.42 have a solution then the maximum value in 5.44 is 0.
We have the following result. Lemma 1:
If 5.41, 5.42, and 5.43 have a solution, then there is an optimal basic feasible solution of 5.44 which is also a solution f 5.41, 5.42, and 5.43.
Proof: Let
ˆ x, ˆ
y, ˆ λ, ˆ
µ be a solution of 5.41, 5.42, and 5.43. Then ˆ x, ˆ
y, ˆ λ, ˆ
µ, ˆ z = 0, ˆ
ξ = 0 is an optimal solution of 5.44. Furthermore, from 5.42 and 5.43 we see that at most
n + m components of
ˆ x, ˆ
y, ˆ λ, ˆ
µ are non-zero. But then a repetition of the proof of Lemma 1 of 4.3.1 will also prove this lemma.
♦ This lemma suggests that we can apply the Simplex algorithm of 4.3.2 to solve 5.44, starting
with the basic feasible solution z = b, ξ = −c, in order to obtain a solution of 5.41, 5.42, and
5.43. However, Step 2 of the Simplex algorithm must be modified as follows to satisfy 5.43: If a variable
x
j
is currently in the basis, do not consider µ
j
as a candidate for entry into the basis; if a variable
y
i
is currently in the basis, do not consider λ
i
as a candidate for entry into the basis. If it not possible to remove the
z
i
and ξ
j
from the basis, stop. The above algorithm is due to Wolfe [1959]. The behavior of the algorithm is summarized below.
Theorem 2: Suppose
P is positive definite. The algorithm will stop in a finite number of steps at an optimal basic feasible solution
ˆ x, ˆ
y, ˆ λ, ˆ
µ, ˆ z, ˆ
ξ of 5.44. If ˆ z = 0 and ˆ
ξ = 0 then ˆ x, ˆ
y, ˆ λ, ˆ
µ solve 5.41, 5.42, and 5.43 and
ˆ x is an optimal solution of 5.39. If ˆ
z 6= 0 or ˆ ξ 6= 0, then there is no
solution to 5.41, 5.42, 5.43, and there is no feasible solution of 5.39. For a proof of this result as well as for a generalization of the algorithm which permits positive
semi -definite
P see Cannon, Cullum, and Polak [1970], p. 159 ff.
5.4 Computational Method
We return to the general NP 5.45, Maximize
f x
subject to f
i
x ≤ 0, i = 1, . . . , m , 5.45
where x ∈ R
n
, f
i
: R
n
→ R, 0 ≤ i ≤ m, are differentiable. Let Ω ⊂ R
n
denote the set of feasible solutions. For
ˆ x ∈ Ω define the function ψˆx : R
n
→ R by ψˆ
xh = max{−f
0x
ˆ xh, f
1
ˆ x + f
1x
ˆ xh, . . . , f
m
ˆ x + f
mx
ˆ xh}.
Consider the problem: Minimize
ψˆ xh
subject to − ψˆxh − f
0x
ˆ xh
≤ 0 , −ψˆxh + f
i
ˆ xf
ix
h ≤ 0 , 1 ≤ i ≤ m , −1 ≤ h
j
≤ 1 , 1 ≤ j ≤ n . 5.46
72
CHAPTER 5. NONLINEAR PROGRAMMING
.
f x = F
x
∗
f x
k
f x = f
x
k
f
2
= 0
f
1
= 0
Ω f
3
= 0 ▽f
2
x
k
x
k
▽f
3
x
k
▽f
1
x
k
▽f x
k
hx
k
Figure 5.8: hx
k
is a feasible direction. Call
hˆ x an optimum solution of 5.46 and let h
ˆ x = ψˆ
xhˆ x be the minimum value at-
tained. Note that by Exercise 1 of 4.5.1 5.46 can be solved as an LP. The following algorithm is due to Topkis and Veinott [1967].
Step 1. Find
x ∈ Ω, set k = 0, and go to Step 2.
Step 2. Solve 5.46 for
ˆ x = x
k
and obtain h
x
k
, hx
k
. If h x
k
= 0, stop, otherwise go to Step 3.
Step 3. Compute an optimum solution
µx
k
to the one-dimensional problem, Maximize
f x
k
+ µhx
k
, subject to
x
k
+ µhx
k
∈ Ω, µ ≥ 0 , and go to Step 4.
Step 4. Set
x
k+1
= x
k
+ µx
k
hx
k
, set k = k + 1 and return to Step 2. The performance of the algorithm is summarized below.
Theorem 1: Suppose that the set
Ωx = {x|x ∈ Ω, f
x ≥ f x
} is compact, and has a non-empty interior, which is dense in
Ωx . Let x
∗
be any limit point of the sequence
x , x
1
, . . . , x
k
, . . . , generated by the algorithm. Then the Kuhn-Tucker conditions are satisfied at
x
∗
. For a proof of this result and for more efficient algorithms the reader is referred to Polak [1971].
Remark: If
h x
k
0 in Step 2, then the direction hx
k
satisfies f
0x
x
k
hx
k
0, and f
i
x
k
+ f
ix
x
K
hx
k
0, 1 ≤ i ≤ m. For this reason hx
k
is called a desirable feasible direction. See Figure 5.8.
5.5.
APPENDIX 73
5.5 Appendix
The proofs of Lemmas 4,7 of Section 2 are based on the following extremely important theorem see Rockafeller [1970].
Separation theorem for convex sets. Let
F, G be convex subsets of R
n
such that the relative interiors of
F, G are disjoint. Then there exists λ ∈ R
n
, λ 6= 0, and θ ∈ R such that λ
′
g ≤ θ for all g ∈ G λ
′
f ≥ θ for all f ∈ F . Proof of Lemma 4:
Since M is stable at ˆb there exists K such that
M b − Mˆb ≤ K|b − ˆb| for all b ∈ B . 5.47
In R
1+m
consider the sets F = {r, b|b ∈ R
m
, r K|b − ˆb|} , G = {r, b|b ∈ B, r ≤ Mb − Mˆb} .
It is easy to check that F, G are convex, and 5.47 implies that F ∩ G = φ. Hence, there exist
λ , . . . , λ
m
6= 0, and θ such that λ
r +
m
X
i=1
λ
i
b
i
≤ θ for r, b ∈ G , λ
r +
m
X
i=1
λ
i
b
i
≥ θ for r, b ∈ F . 5.48
From the definition of F , and the fact that λ
, . . . , λ
m
6= 0, it can be verified that 5.49 can hold only if
λ 0. Also from 5.49 we can see that
m
X
i=1
λ
i
ˆb
i
≥ θ, whereas from 5.48
m
X
i=1
λ
i
ˆb
i
≤ θ, so that
m
X
i=1
λ
i
ˆb
i
= θ. But then from 5.48 we get M b − Mˆb ≤
1 λ
[θ −
m
X
i=1
λ
i
b
i
] =
m
X
i=1
− λ
i
λ b
i
− ˆb. ♦
Proof of Lemma 7: Since ˆ
b is in the interior of B, there exists ε 0 such that b ∈ B whenever |b − ˆb| ε .
5.49 In
R
1+m
consider the sets F = {r, ˆb|r Mˆb}
G = {r, b|b ∈ B, r ≤ Mb} . Evidently,
F, G are convex and F ∩ G = φ, so that there exist λ , . . . , λ
m
6= 0, and θ such that λ
r +
m
X
i=1
λ
i
ˆb
i
≥ θ , for r Mˆb , 5.50
74
CHAPTER 5. NONLINEAR PROGRAMMING
λ r +
m
X
i=1
λ
i
ˆb
i
≤ θ , for r, b ∈ G . 5.51
From 5.49, and the fact that λ
, . . . , λ
m
6= 0 we can see that 5.50 and 5.51 imply λ 0.
From 5.50,5.51 we get λ
M ˆb +
m
X
i=1
λ
i
ˆb
i
= θ , so that 5.52 implies
M b ≤ ˆb +
m
X
i=1
− λ
i
λ b
i
− ˆb
i
. ♦
Chapter 6
SEQUENTIAL DECISION PROBLEMS: DISCRETE-TIME OPTIMAL
CONTROL
In this chapter we apply the results of the last two chapters to situations where decisions have to be made sequentially over time. A very important class of problems where such situations arise is in
the control of dynamical systems. In the first section we give two examples, and in Section 2 we derive the main result.
6.1 Examples
