4.3.
THE SIMPLEX ALGORITHM 37
∂M
+
∂c
j
ˆb, ˆ c ≥ ˆx
j
≥
∂M
−
∂c
j
ˆb, ˆ c , 1 ≤ j ≤ n ,
4.23 Proof:
We first show 4.22, 4.23 assuming that the partial derivatives exist. By strong duality M ˆb, ˆ
c = ˆ λ
′
ˆb, and by weak duality Mˆbi, ε, ˆc ≤ ˆλ
′
ˆbi, ε, so that
1 ε
{Mˆbi, ε, ˆc − Mˆb, ˆc} ≤
1 ε
ˆ λ
′
{ˆbi, ε − ˆb}ˆλ
i
, for ε 0,
1 ε
{Mˆb, ˆc − Mˆbi, −ε, ˆc} ≥
1 ε
ˆ λ
′
{ˆb − ˆbi, −ε} = ˆλ
i
, for ε 0. Taking limits as
ε → 0, ε 0, gives 4.22. On the other hand,
M ˆb, ˆ c = ˆ
c
′
ˆ x, and M ˆb, ˆ
cj, ε ≥ ˆcj, ε
′
ˆ x, so that
1 ε
{Mˆb, ˆcj, ε − Mˆb, ˆc} ≥
1 ε
{ˆcj, ε
′
− ˆc}
′
ˆ x = ˆ
x
j
, for ε 0,
1 ε
{Mˆb, ˆc − Mˆb, ˆcj, −ε} ≤
1 ε
{ˆc − ˆcj, −ε}
′
ˆ x = ˆ
x
j
, for ε 0, which give 4.23 as
ε → 0, ε 0. Finally, the existence of the right and left partial derivatives follows from Exercises 8, 9 below.
♦ We recall some fundamental definitions from convex analysis.
Definition: X ⊂ R
n
is said to be convex if x, y ∈ X and 0 ≤ θ ≤ 1 implies θx+1−θy ∈ X.
Definition: Let
X ⊂ R
n
and f : X → R. i f is said to be convex if X is convex, and x, y ∈ X,
0 ≤ θ ≤ 1 implies fθx + 1 − θy ≤ θf x + 1 − θf y. ii f is said to be concave if −f is convex, i.e.,
x, y ∈ X, 0 ≤ θ ≤ 1 implies f θx + 1 − θy ≥ θf x + 1 − θfy.
Exercise 8: a Show that
Ω
p
, Ω
d
, and the sets B ⊂ R
m
, C ⊂ R
n
defined above are convex sets. b Show that for fixed
c ∈ C, M·, c : B → R is concave and for fixed b ∈ B, Mb, · : C → R is convex.
Exercise 9:
Let X ⊂ R
n
, and f : X → R be convex. Show that at each point ˆx in the interior of
X, the left and right hand partial derivatives of f exist. Hint: First show that for ε
2
ε
1
0 δ
1
δ
2
, 1ε
2
{fˆxi, ε
2
− f ˆx} ≥ 1ε
1
{f ˆxi, ε
1
− f ˆx} ≥ 1δ
1
{f ˆxi, δ
1
− f ˆx} ≥ 1δ
2
{f ˆxi, δ
2
− f ˆx}. Then the result follows immediately. Remark 1:
Clearly if ∂M∂b
i
ˆb exists, then we have equality in 4.22, and then this result compares with 3.14.
Remark 2: We can also show without difficulty that
M ·, c and Mb, · are piecewise linear more accurately, linear plus constant functions on
B and C respectively. This is useful in some computational problems.
Remark 3: The variables of the dual problem are called Lagrange variables or dual variables or
shadow-prices. The reason behind the last name will be clear in Section 4.
4.3 The Simplex Algorithm
4.3.1 Preliminaries
We now present the celebrated Simplex algorithm for finding an optimum solution to any LP of the form 4.24:
Maximize subject to
c
1
x
1
+ . . . + c
n
x
n
a
il
x
1
+ . . . + a
in
x
n
= b
i
, x
j
≥ 0 , 1 ≤ i ≤ m
1 ≤ j ≤ n . 4.24
38
CHAPTER 4. LINEAR PROGRAMMING
As mentioned in 4.1 the algorithm rests upon the observations that if an optimal exists, then at least one vertex of the feasible set
Ω
p
is an optimal solution. Since Ω
p
has only finitely many vertices see Corollary 1 below, we only have to investigate a finite set. The practicability of this investigation
depends on the ease with which we can characterize the vertices of Ω
p
. This is done in Lemma 1. In the following we let
A
j
denote the jth column of A, i.e., A
j
= a
1j
, . . . , a
mj ′
. We begin with a precise definition of a vertex.
Definition: x ∈ Ω
p
is said to be a vertex of Ω
p
if x = λy + 1 − λz, with y, z in Ω
p
and 0 λ 1, implies x = y = z.
Definition: For
x ∈ Ω
p
, let Ix = {j|x
j
0}. Lemma 1:
Let x ∈ Ω
p
. Then x is a vertex of Ω
p
iff {A
j
|j ∈ Ix} is a linearly independent set.
Exercise 1:
Prove Lemma 1. Corollary 1:
Ω
p
has at most
m
X
j=1
n n − j
vertices. Lemma 2:
Let x
∗
be an optimal decision of 4.24. Then there is a vertex z
∗
of Ω
p
which is optimal. Proof:
If {A
j
|j ∈ Ix
∗
} is linearly independent, let z
∗
= x
∗
and we are done. Hence suppose {A
j
|j ∈ Ix
∗
} is linearly dependent so that there exist γ
j
, not all zero, such that X
j∈Ix
∗
γ
j
A
j
= 0 . For
θ ∈ R define zθ ∈ R
n
by z
j
θ = x
∗ j
= θγ
j
, x
∗ j
= 0 ,
j ∈ Ix
∗
j 6∈ Ix
∗
. Azθ =
X
j∈Ix
∗
z
j
θA
j
= X
j∈Ix
∗
x
∗ j
A
j
+ θ X
j∈Ix
∗
γ
j
A
j
= b + θ · 0 = b . Since
x
∗ j
0 for j ∈ Ix
∗
, it follows that zθ ≥ 0 when |θ| ≤ min
n
x
∗ j
|γ
j
|
j ∈ Ix
∗
o = θ
∗
say .
Hence zθ ∈ Ω
p
whenever |θ| ≤ θ
∗
. Since x
∗
is optimal we must have c
′
x
∗
≥ c
′
zθ = c
′
x
∗
+ θ X
j∈Ix
∗
c
j
y
j
for −
∗
θ ≤ θ ≤ θ
∗
. Since
θ can take on positive and negative values, the inequality above can hold on if X
J∈Ix
∗
c
j
γ
j
= 0, and then c
′
x
∗
= c
′
zθ, so that zθ is also an optimal solution for |θ| ≤ θ
∗
. But from the definition of
zθ it is easy to see that we can pick θ with
|θ | = θ
∗
such that z
j
θ = x
∗ j
+θ γ
j
= 0 for at least one
j = j in
Ix
∗
. Then, Izθ
⊂ Ix
∗
− {j } .
4.3.
THE SIMPLEX ALGORITHM 39
Again, if {A
j
|j ∈ Izθ } is linearly independent, then we let z
∗
= zθ and we are done.
Otherwise we repeat the procedure above with zθ
. Clearly, in a finite number of steps we will find an optimal decision
z
∗
which is also vertex. ♦
At this point we abandon the geometric term “vertex” and how to established LP terminology. Definition:
i z is said to be a basic feasible solution if z ∈ Ω
p
, and {A
j
|j ∈ Iz} is linearly independent. The set
Iz is then called the basis at z, and x
j
, j ∈ Iz, are called the basic
variables at z .
x
j
, j 6∈ Iz are called the non-basic variables at z.
Definition: A basic feasible solution
z is said to be non-degenerate if Iz has m elements. Notation:
Let z be a non-degenerate basic feasible solution, and let j
1
j
2
. . . j
m
constitute Iz. Let Dz denote the m × m non-singular matrix Dz = [A
j
1
.. .
A
j
2
.. .
. . . ..
. A
j
m
], let cz denote the m-dimensional column vector cz = c
j
1
, . . . , c
j
m
′
and define λz by λ
′
z = c
′
z[Dz]
−1
. We call λz the shadow-price vector at z.
Lemma 3: Let
z be a non-degenerate basic feasible solution. Then z is optimal if and only if λ
′
zA ≥ c
j
, for all , j 6∈ Iz . 4.25
Proof: By Exercise 6 of Section 2.2,
z is optimal iff there exists λ such that λ
′
A
j
= c
j
, for , j ∈ Iz , 4.26
λ
′
A
j
≥ c
j
, for , j 6∈ Iz , 4.27
But since z is non-degenerate, 4.26 holds iff λ = λz and then 4.27 is the same as 4.25.
♦
4.3.2 The Simplex Algorithm.
