Directory UMM :Data Elmu:jurnal:O:Operations Research Letters:Vol27.Issue1.2000:
Operations Research Letters 27 (2000) 29–37
www.elsevier.com/locate/dsw
On improved Choi–Goldfarb solution-containing ellipsoids
in linear programming
Igor S. Litvinchev 1
Computing Center, Russian Academy of Sciences, Vavilov 40, 117967 Moscow, Russia
Received 1 May 1998; received in revised form 1 February 2000
Abstract
Ellipsoids that contain all optimal primal solutions, those that contain all optimal dual slack solutions, and primal–
dual ellipsoids are derived. They are independent of the algorithm used and have a smaller size than the Choi–Goldfarb
c 2000 Elsevier Science B.V. All rights reserved.
ellipsoids [J. Optim. Theory Appl. 80 (1994) 161–173].
Keywords: Linear programming; Interior point methods; Containing ellipsoids
1. Introduction
Unlike the simplex method for linear programming, which moves from one basic feasible solution to another
basic feasible solution, interior algorithms create a sequence of interior points converging to an optimal feasible
solution. To identify optimal basic and non-basic variables during the course of the algorithm several criteria
have been proposed, associated with the interior point techniques used. These criteria were obtained by
constructing ellipsoids which contain all optimal primal solutions and=or ellipsoids which contain all optimal
dual slack solutions, and are all algorithm dependent (see e.g. [2,3,5,7–9]). Choi and Goldfarb [1] have further
developed this approach, deriving solution containing ellipsoids which are independent of the interior point
algorithm used, but larger than algorithm-dependent ellipsoids. Concerning solution-containing ellipsoids we
also note that the rst two-sided ellipsoidal approximation of polyhedra based on primal barrier functions is
due to Sonnevend [4].
In this paper we propose the approach to diminish the size of the solution-containing ellipsoids derived
in [1]. To get the tighter estimation of the ellipsoid size a certain localization of the optimal solution set is
E-mail address: [email protected] (I.S. Litvinchev).
The research supported in part by the grants from FAPESP, Brazil (No. 1997=7153-2) and RFBR, Russia (No. 99-01-01071).
1
c 2000 Elsevier Science B.V. All rights reserved.
0167-6377/00/$ - see front matter
PII: S 0 1 6 7 - 6 3 7 7 ( 0 0 ) 0 0 0 2 6 - 2
30
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
used. It is shown that the proper choice of the localization results in a smaller containing ellipsoid. A similar
approach is used to derive the primal–dual containing ellipsoids.
The paper is organized as follows. In Section 2, some basic notations and results, associated with Choi–
Goldfarb ellipsoids are stated. In Section 3, the improved primal ellipsoids are constructed. In Section 4,
the primal–dual ellipsoids are considered, and a new criterion for identifying the optimal basis is stated and
compared with the previous results.
2. Denitions and notation
Consider the primal–dual pair of linear programs:
min{ct x | Ax = b; x¿0};
(1)
max{bt y | At y + z = c; z¿0};
(2)
where A ∈ Rm×n and b; c; x; y and z are suitably dimensioned. Let x∗ and (z ∗ ; y∗ ) be optimal solutions to
(1) and (2), respectively. It is assumed that the optimal solution sets for both (1) and (2) are bounded, or
equivalently, strictly feasible primal and dual solutions exist, and that rank(A) = m: The upper case letters X
and Z are used to denote diagonal matrices whose elements are the elements of the vectors x and z, and k · k
denotes the Euclidean norm.
It was established in [1] that if x and (z;
y)
are primal and dual feasible solutions, then the ellipsoid
− x)k6};
Ep = {x ∈ Rn | kZ(x
where
2 + 2 ;
2 = kZ xk
= x t z
contains all optimal primal solutions x∗ : Here the “ellipsoid” is understood in a general sense allowing also
unbounded sets.
Using this ellipsoid the lower bound xi− () for xi∗ was derived by the solution of the problem
− x)k6;
Ax = b; ct x6ct x}:
min{xi | kZ(x
If z ¿ 0 (which will be assumed valid throughout the paper) then the optimal solution xi− () of this problem
takes the following form. Let ei denote the ith unit vector and
= Z
R(Z)
−1
[I − Z
−1 t
A (AZ
−2 t −1
A)
AZ
−1
]Z
−1
:
Then
xi− () = xi −
q
xi− () = xi −
q
i
eit R(Z)e
and
i − (ct R(Z)e
i )2 =ct R(Z)c
eit R(Z)e
i ¡0
for ct R(Z)e
i ¿0:
for ct R(Z)e
Respectively, the following criterion for identifying optimal basic variables was obtained: if xi− () ¿ 0; then
the ith primal variable is an optimal basic variable, i.e., xi∗ ¿ 0 and zi∗ = 0:
We see that the smaller the size of the ellipsoid Ep ; the larger is the value of xi− (): Below we consider
the approach to diminish the size of the solution-containing ellipsoid.
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
31
3. Improved primal solution-containing ellipsoid
Denote by P ∗ the optimal solution set of (1) and let W be a closed bounded subset of Rn such that
W ⊇ P ∗ : We will refer to such a W as a primal localization. Let S ∈ Rn×n be a diagonal matrix having
positive diagonal entries si :
For any x ∈ P ∗ we have
2 − 2x t (S Z)
2x
− x)k
2 + kS Zxk
kS Z(x
2 = kS Z xk
2 − 2x t (S Z)
2 w}
6 kS Z xk
2 + max{kS Zwk
w∈W
≡ 2 (S; W );
(3)
where the inequality follows from the condition x ∈ P ∗ ⊆ W:
Then the ellipsoid
− x)k6(S;
W )}
Ep (S; W ) = {x ∈ Rn | kS Z(x
contains all optimal primal solutions.
Using the ellipsoid Ep (S; W ) the lower bound xi− can be derived similarly as in Section 2. It is not hard
for
to verify that we need only to substitute R(Z)
−1 At (A(S Z)
−2 At )−1 A(S Z)
−1 ](S Z)
−1 :
= (S Z)
−1 [I − (S Z)
R(S Z)
and use (S; W ) instead of :
The reason for using the scaled ellipsoid is as follows. Suppose for example that the pair (x;
z)
was produced
takes the form
by a primal–dual method [6]. Let S = (XZ)−1=2 : For this case R(S Z)
R(D−1 ) = D[I − DAt (AD2 At )−1 AD]D;
1=2 −1=2
: Note that the matrix AD2 At is computed and decomposed into its Cholesky factor in
where D = X Z
most of the primal–dual methods.
To dene a localization W we can use some constraints of the primal problem or a linear combination of
them. Below we consider the localization which results in a closed form expression for (S; W ):
Proposition 1. Let the primal localization be dened by W1 = {w ∈ Rn | z t w6; w¿0}: Then
2 + max{si2 ( − 2zi xi )}:
2 (S; W1 ) = kS Z xk
i
Proof. For any x ∈ P ∗ we have
t x = ct x − y t b6ct x − y t b = z t x =
z t x = (c − At y)
and hence P ∗ ⊆ W1 : Moreover since z ¿ 0 then W1 is bounded, i.e., W1 is a localization. To estimate (S; W1 )
in (3) we should solve the problem
2 − 2x t (S Z)
2 w}:
f∗ = max {kS Zwk
w∈W1
The objective function of this problem is strictly convex and hence the optimal solution is attained at a vertex
of the non-degenerate simplex W1 : The non-zero component of the ith vertex is = zi and thus
f∗ = max{si2 ( − 2zi xi )};
i
which by (3) gives the required expression for (S; W1 ):
32
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
Note that if S = I and x ¿ 0; then
2 + 2 − 2 min zi xi ¡ 2
2 (I; W1 ) = kZ xk
i
and hence Ep (I; W1 ) ⊂ Ep :
The other choice of the scaling matrix S associated with the primal–dual algorithms is S = (XZ)−1=2 : For
this case, we have
2 ((XZ)−1=2 ; W1 ) =
2
− :
mini zi xi
A similar approach can be used to derive a dual ellipsoid that contains all optimal dual slack solutions, i.e.
V )};
Ed (S; V ) = {z ∈ Rn | kS X (z − z)k6(S;
where
2 + max{kS X k2 − 2z t (S X )2 }:
2 (S; V ) = kS X zk
∈V
A dual localization V is a closed bounded subset of Rn such that z ∗ ∈ V for any optimal dual slack solution
z ∗ : It is not hard to verify that if x ¿ 0; then V1 = { ∈ Rn | x t 6; ¿0} is the dual localization and
(S; V1 ) = (S; W1 ): The ellipsoid Ed (I; V1 ) is smaller than the dual-containing ellipsoid Ed = {z ∈ Rn | kX (z −
z)k6}
considered in [1] and it can be used similarly to identify optimal non-basic primal variables.
4. Primal–dual solution-containing ellipsoid
Denote by F ∗ ⊆ R2n the set of all optimal primal–dual pairs (x∗ ; z ∗ ) and let
be a closed bounded subset
of R2n such that
⊇ F ∗ : We will refer to such an
as a primal–dual localization. In this section we assume
(x;
z)
¿ 0:
For any (x; z) ∈ F ∗ we have similar to (3):
2 + kS X k2
− x)k
2 + max {kS Zwk
2 6 2kS Z xk
kS Z(x
2 + kS X (z − z)k
(w;)∈
2 w − 2z t (S X )2 } ≡ 2 (S;
)
− 2x t (S Z)
and hence the ellipsoid
− x)k
2 + kS X (z − z)k
2 62 (S;
)}
Epd (S;
) = {(x; z) ∈ R2n | kS Z(x
contains all optimal primal–dual pairs (x∗ ; z ∗ ):
Proposition 2. Let the primal–dual localization be dened by
1 = {(w; ) ∈ R2n | z t w + xt = ; (w; )¿0}:
Then
2 + max{si2 ( − 2zi xi )}:
2 (S;
1 ) = 2kS Z xk
i
t (z − z)
= 0 and xt z = 0 by the complementarity
Proof. For any (x; z) ∈ F ∗ we have z t x + x t z = since (x − x)
∗
condition. Hence F ⊆
1 and
1 is a localization. Then the required expression for (S;
1 ) is obtained
similar to the proof of Proposition 1.
Now we use Epd to identify, if possible, the optimal basis. Consider the problem
qt x → min;
(4)
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
Ax = b;
At y + z = c;
33
(5)
− x)k
2 62 ;
kS Z(x
2 + kS X (z − z)k
(6)
z t x + x t z = ;
(7)
x t z6;
(8)
z t x6:
(9)
Here q is such that the objective of this problem coincides with xi up to a positive scale factor. In [9] the
problem similar to (4) – (6) was discussed for q = Dei :
−1 q; d−2 = ke v k2 + kek
2 − ke u k2 ; where v ; u are used to denote the projections of
Let e = S −1 e; q = (S Z)
n
−1 and AS X , respectively. Denote also
∈ R onto the null space of the matrices A(S Z)
−1
(e t qv )2
t
2
2
2
4
2
:
− ke u k ) ke v k kqv k −
B = (e qv ) (kek
ke v k2
Proposition 3. The optimal solution x− of problem (4)–(9) is such that
(a) if e t qv ¿0 and kqv k ¡ (e t qv ); then
s
2
e t qv
(e t qv )2
t −
t
2−
2−
−
k
k
q
;
q xa = q x −
v
ke v k2
ke v k2
ke v k2
2 − ke u k2 ) − 2 ]; then
(b) if e t qv 60 and 2 ¡ B[2 (kek
s
2
(e t qv )2
t −
t
2
2
−
kqv k −
;
q xb = q x −
kek
2 − ke u k2
ke v k2
(c) else
qt xc− = qt x − (e t qv )d2 −
q
(2 − 2 d2 )(kqv k2 − d2 (e t qv )2 ):
Proof. The KKT optimality conditions for (4) – (9) give
2 (x − x)
− ( − )z = 0;
q + At u + 2(S Z)
(10)
− ( − )x = 0;
v + 2(S X )2 (z − z)
(11)
Av = 0;
¿ 0;
¿0;
¿0;
where ; ; and are the Lagrange multipliers for the restrictions (7) – (9).
−2 and utilizing the condition A(x − x)
= 0 we get
Multiplying (10) by A(S Z)
−1 (( − )e − q):
−2 At ]−1 A(S Z)
u = [A(S Z)
−2 and substituting the expression for u we obtain
Multiplying (10) by (S Z)
1
−1 [( − )e − q]
v:
(S Z)
2
+ (z − z)
= 0 we get
Now multiplying (11) by A and using the conditions Av = 0; At (y − y)
x − x =
y − y = −
−
−2 At ]−1 Ax
[A(S Z)
2
34
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
and hence
=
z − z = −At (y − y)
− −1
(S X ) (e − e u ):
2
+ z t (x − x)
= −: Substituting x − x and z − z in this expression we obtain the
By (7) we have x t (z − z)
equation for which gives
2 − ke u k2 ) − 2]:
= d2 [e t qv + ke v k2 + (kek
If ¿ 0; then by the complementarity condition for (8) we have x t z=: Hence by (7) it follows 0= z t x ¡
and = 0 by the complementarity condition for (9). Similarly if ¿ 0, then = 0. We can write both these
conditions in the form = 0.
= 0 and
Remembering that = x t z the complementarity conditions for (8) and (9) take the form x t (z − z)
= 0. Substituting the expressions for x − x and z − z we obtain
z t (x − x)
( − ) = 0;
[( − )ke v k2 − e t qv ] = 0:
Utilizing the expression for and the condition = 0 we get
d2 (e t qv − ke v k2 − 2) = 0;
(12)
kek
2 − ke u k2
2
2
2
2
= 0:
−
d
(k
ek
−
k
e
k
)
−
2d
−e t qv
u
ke v k2
(13)
If e t qv 60 then the term in the brackets in (12) is strictly negative and hence = 0. By (13) we have that
if e t qv ¿0 then = 0.
Since the quadratic constraint (6) must be active, then
2
2
−
1
k( − )e v − qv k2 +
ke − e u k2 :
2 =
2
2
Substituting the expression for we obtain after the algebraic manipulations:
2
2
1
1
2
2 t
2
2
2 2
(kqv k − d (e qv ) ) −
d2 {(e t qv )( − )(kek
2 − ke u k2 )
= d +
2
2
2 − ke u k2 )]}:
+ 2[ke v k2 + (kek
(14)
Now we are in a position to derive the main results of the proposition.
Suppose that e t qv ¿0 and kqv k ¡ (e t qv ). By (13) it follows that = 0. Let ¿ 0. Then by (12) we
have
=
e t qv − 2
:
ke v k2
Substituting this in (14) with = 0 we obtain
2
2 ke v k2 − 2
1
=
:
2
kqv k2 ke v k2 − (e t qv )2
(15)
Note that since kqv k ¡ (e t qv ), then the numerator of the above expression is strictly positive. Using the
expression for it is not hard to verify that in (15) is strictly positive. Now utilizing the expressions for
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
x − x;
and we have
=
qt (x − x)
1
e t qv
1
[e t qv − kqv k2 ] = −
−
2
ke v k2
2
kqv k2 −
(e t qv )2
ke v k2
35
;
which nally proves part (a) of the proposition.
Note that if e t qv ¿0 and kqv k¿(e t qv ) then the case ¿ 0 is impossible and we have = = 0.
2 − ke u k2 ) − 2 ]. By (12) it follows that = 0. Let ¿ 0.
Suppose now that e t qv 60 and 2 ¡ B[2 (kek
Then by (13) we have
2 − ke u k2 ) + 2ke v k2
e t qv (kek
:
ke v k2 (kek
2 − ke u k2 )
Substituting this in (14) with = 0 we obtain
2
2 (kek
2 − ke u k2 ) − 2
1
ke v k2 :
=
2
(kek
2 − ke u k2 )(kqv k2 ke v k2 − (e t qv )2 )
=−
(16)
Note that under our assumption the numerator of this expression is strictly positive. Moreover, if we substitute
this in (16), then ¿ 0.
Substituting x − x;
and we have
s
2
1
(e t qv )2
t
2
t
2−
= [( − )e qv − kqv k ] = −
k
2 −
k
q
;
q (x − x)
v
2
kek
2 − ke u k2
ke v k2
which proves part (b) of the proposition.
2 − ke u k2 ) − 2 ] then the case ¿ 0 is impossible and = = 0.
Note that if e t qv 60 and 2 ¿B[2 (kek
If either (a) or (b) is not satised, then = = 0. For this case we have = d2 (e t qv − 2) and
=
qt (x − x)
d2 (e t qv − 2) t
1
1
(e t qv − kqv k2 ) =
e qv − kqv k2
2
2
2
1
(kqv k2 − d2 (e t qv )2 ):
2
Obtaining from (14) with = = 0 we get the expression for qt x− required in part (c) of the proposition.
= −d2 2 e t qv −
Since (x∗ ; z ∗ ; y∗ ) is feasible to (4) – (9) for = (S;
1 ), we have the following criterion for identifying the
optimal basis: if qt x− ¿ 0, then xi∗ ¿ 0 and zi∗ = 0.
Note that for part (c) of Proposition 3 we have = = 0 and hence constraints (8) and (9) can be
relaxed without changing the optimal value. For parts (a) and (b), either (8) or (9) is active and hence
qt xc− 6min{qt xa− ; qt xb− }.
In [9] the primal–dual containing ellipsoid centered at the origin was considered for S = (XZ)−1=2 and
(x;
z;
y)
∈ N2 (
), where
∈ [0; 1),
0
N2 (
) = (x; y; z) ∈ F | XZe − e 6
n
n
and F 0 is the set of all strictly feasible primal–dual solutions.
For (x;
z;
y)
∈ N2 (
) we have zi xi ¿(1 −
)=n; i = 1; n and by Proposition 2
2
n
2
6
≡ ;
mini zi xi 1 −
such that the size of our ellipsoid is the same as in [9]. Similarly we can estimate the size of the ellipsoid
for (x;
z;
y)
∈ N−∞ () = {(x; z; y) ∈ F 0 | zi xi ¿=n; i = 1; n}; ∈ (0; 1).
2 ((XZ)−1=2 ;
1 ) =
36
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
−
The lower bound qt xYT
obtained in [9, Theorem 3] by the solution of the problem similar to (4) – (6) for
−1
q = D ei is as follows:
q
2
t −
t
q xYT = ei P e − ( − )(1 − p ii );
where P = Dt At (AD2 At )−1 AD and p ii is the ith diagonal entry of P.
−1=2
2
−1
, part (c) of Proposition 3 gives
Since u = v = (I − P) and d = for S = (XZ)
s
1
2
−
e ]2 ¿ qt xYT
:
qt xc− = eit P e − ( − ) 1 − p ii − [eit (I − P)
Hence Proposition 3 results in the stronger criterion for identifying the optimal basis.
It is interesting to compare the lower bound obtained in Proposition 3 with the bound xi− () in Section 2.
We have
2 + 2 − 2 min zi xi = 2 + kZ xk
2 − 2 min zi xi :
2 (I;
1 ) = 2kZ xk
i
i
If (x;
z;
y)
∈ N2 (
), then (1 −
)=n6zi xi 6(1 +
)=n and
kZ xk
2 − 2 min zi xi 62 (
2 + 4
− 1)=n:
i
√
For
∈ [0; 5 − 2) the right-hand side of the latter inequality is strictly negative and (I;
1 ) ¡ .
= Z −1 (Z −1 )v for any ∈ Rn and (Z −1 c)v = ev by At y + z = c, it is not hard to verify that
Since R(Z)
xi− () coincides with the optimal solution of the following problem:
− x)k6;
Ax = b; z t x6}:
xi− () = min{xi | kZ(x
(17)
√
z;
y)
∈ N2 (
),
∈ [0; 5 − 2), = (I;
1 ) ¡ ,
Comparing (17) with the problem (4) – (9) for q = ei , S = I; (x;
we obtain xi− () ¡ eit x− that results in improved criterion for identifying the optimal basis.
The ellipsoids considered in this paper are algorithm-independent, that is they do not depend on how the
primal–dual pair (x;
z)
was obtained. Meanwhile the size of the ellipsoid depends on the proximity of (x;
z)
to
the central path: the closer to the central path, the smaller the size. This is one more reason for maintaining
(x;
z)
in the neighborhood of the central path during the course of the interior algorithm.
Acknowledgements
The author gratefully acknowledges two anonymous referees for their helpful comments and remarks.
Appendix
Since our expression for xi− () is slightly dierent from the one used in [1, Criterion (C4), p. 170], we
give here the complete derivation for xi− ().
Proposition A1. The optimal solution xi− () of the problem
2
2
t Z (x − x)6
; Ax = b; ct x6ct x}
min{eit x | (x − x)
(A.1)
is as follows:
xi− () = xi −
q
i − (ct R(Z)e
i )2 =ct PAZ −1 c
eit R(Z)e
i ¡0
for ct R(Z)e
37
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
and
xi− ()
q
i
= xi − eit R(Z)e
i ¿0:
for ct R(Z)e
Proof. The Lagrangian, associated with (A.1) is
2
t Z (x − x)
− 2 ] + ut ( Ax − b) + (ct x − ct x):
L = eit x + [(x − x)
The KKT condition @L=@x = 0 gives
2
+ c + At u = 0:
ei + 2Z (x − x)
(A.2)
2
− ] = 0, (c x − c x)
= 0, A(x − x)
=0
Multiplying (A.2) by (x − x)
and utilizing conditions [(x − x)
Z (x − x)
we obtain
= (xi − xi )=22 :
Multiplying (A.2) by AZ
−2 t −1
u = −(AZ
A)
−2
AZ
t
2
t
t
we get
−2
(ei + c):
From (A.2) we have
−2
i + c):
2(x − x) = Z (At u + ei + c) = R(Z)(e
(A.3)
t
Multiplying (A.3) by c we obtain
i + ct R(Z)c:
062ct (x − x) = ct R(Z)e
(A.4)
If c R(Z)ei ¡ 0, then it follows from (A.4) that ¿ 0 and hence by the complementarity condition
i =ct R(Z)c.
Substituting and in (A.3) and multiplyct (x − x) = 0. From (A.4) we have = − ct R(Z)e
ing by ei we obtain the rst part of Proposition A1.
i ¿0. Suppose that we have relaxed the restriction ct x6ct x in (A.1) or, just the same, put = 0
Let ct R(Z)e
in the Lagrangian. Then similar to (A.3) we obtain for the solution of the relaxed problem
i
2(x − x) = R(Z)e
t
i ¿0 we have ct x6ct x.
and since ¿ 0 and ct R(Z)e
Since the relaxed solution is feasible to (A.1), then it is
optimal to (A.1). This immediately gives the required expression for xi− ().
Note that in [1, Criterion (C4), p. 170] there is a little
aw. The rst expression for xi− () in Proposition A1
i ¿ 0.
was associated there with the case ct R(Z)e
References
[1] I.C. Choi, D. Goldfarb, On solution-containing ellipsoids in linear programming, J. Optim. Theory Appl. 80 (1994) 161–173.
[2] J.L. Gon, J.P. Vial, On the computation of weighted analytic centers and dual ellipsoids with the projective algorithm, Math.
Programming 60 (1993) 81–92.
[3] M. Kojima, Determining basic variables of optimal solutions in Karmarkar’s new LP algorithm, Algorithmica 1 (1986) 499–515.
[4] G. Sonnevend, An ‘Analytical Centre’ for Polyhedrons and New Classes of Global Algorithms for Linear (Smooth, Convex)
Programming, in: Lecture Notes in Control and Information Sciences, Vol. 84, A. Prekopa, J. Szelezsan and B. Strazicky (Eds.),
Springer, Berlin, 1986, pp. 866 –875.
[5] M.J. Todd, Improved bounds and containing ellipsoids in Karmarkar’s linear programming algorithm, Maths. Oper. Res. 13 (1988)
650–659.
[6] S.J. Wright, Primal–Dual Interior-Point Methods, SIAM Publications, Philadelphia, PA, 1997.
[7] Y. Ye, Recovering the optimal basis in Karmarkar’s polynomial algorithm for linear programming, Math. Oper. Res. 15 (1990)
564–572.
[8] Y. Ye, A “build-down” scheme for linear programming, Math. Programming 46 (1990) 61–72.
[9] Y. Ye, M.J. Todd, Containing and shrinking ellipsoids in the path-following algorithm, Math. Programming 47 (1990) 1–9.
www.elsevier.com/locate/dsw
On improved Choi–Goldfarb solution-containing ellipsoids
in linear programming
Igor S. Litvinchev 1
Computing Center, Russian Academy of Sciences, Vavilov 40, 117967 Moscow, Russia
Received 1 May 1998; received in revised form 1 February 2000
Abstract
Ellipsoids that contain all optimal primal solutions, those that contain all optimal dual slack solutions, and primal–
dual ellipsoids are derived. They are independent of the algorithm used and have a smaller size than the Choi–Goldfarb
c 2000 Elsevier Science B.V. All rights reserved.
ellipsoids [J. Optim. Theory Appl. 80 (1994) 161–173].
Keywords: Linear programming; Interior point methods; Containing ellipsoids
1. Introduction
Unlike the simplex method for linear programming, which moves from one basic feasible solution to another
basic feasible solution, interior algorithms create a sequence of interior points converging to an optimal feasible
solution. To identify optimal basic and non-basic variables during the course of the algorithm several criteria
have been proposed, associated with the interior point techniques used. These criteria were obtained by
constructing ellipsoids which contain all optimal primal solutions and=or ellipsoids which contain all optimal
dual slack solutions, and are all algorithm dependent (see e.g. [2,3,5,7–9]). Choi and Goldfarb [1] have further
developed this approach, deriving solution containing ellipsoids which are independent of the interior point
algorithm used, but larger than algorithm-dependent ellipsoids. Concerning solution-containing ellipsoids we
also note that the rst two-sided ellipsoidal approximation of polyhedra based on primal barrier functions is
due to Sonnevend [4].
In this paper we propose the approach to diminish the size of the solution-containing ellipsoids derived
in [1]. To get the tighter estimation of the ellipsoid size a certain localization of the optimal solution set is
E-mail address: [email protected] (I.S. Litvinchev).
The research supported in part by the grants from FAPESP, Brazil (No. 1997=7153-2) and RFBR, Russia (No. 99-01-01071).
1
c 2000 Elsevier Science B.V. All rights reserved.
0167-6377/00/$ - see front matter
PII: S 0 1 6 7 - 6 3 7 7 ( 0 0 ) 0 0 0 2 6 - 2
30
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
used. It is shown that the proper choice of the localization results in a smaller containing ellipsoid. A similar
approach is used to derive the primal–dual containing ellipsoids.
The paper is organized as follows. In Section 2, some basic notations and results, associated with Choi–
Goldfarb ellipsoids are stated. In Section 3, the improved primal ellipsoids are constructed. In Section 4,
the primal–dual ellipsoids are considered, and a new criterion for identifying the optimal basis is stated and
compared with the previous results.
2. Denitions and notation
Consider the primal–dual pair of linear programs:
min{ct x | Ax = b; x¿0};
(1)
max{bt y | At y + z = c; z¿0};
(2)
where A ∈ Rm×n and b; c; x; y and z are suitably dimensioned. Let x∗ and (z ∗ ; y∗ ) be optimal solutions to
(1) and (2), respectively. It is assumed that the optimal solution sets for both (1) and (2) are bounded, or
equivalently, strictly feasible primal and dual solutions exist, and that rank(A) = m: The upper case letters X
and Z are used to denote diagonal matrices whose elements are the elements of the vectors x and z, and k · k
denotes the Euclidean norm.
It was established in [1] that if x and (z;
y)
are primal and dual feasible solutions, then the ellipsoid
− x)k6};
Ep = {x ∈ Rn | kZ(x
where
2 + 2 ;
2 = kZ xk
= x t z
contains all optimal primal solutions x∗ : Here the “ellipsoid” is understood in a general sense allowing also
unbounded sets.
Using this ellipsoid the lower bound xi− () for xi∗ was derived by the solution of the problem
− x)k6;
Ax = b; ct x6ct x}:
min{xi | kZ(x
If z ¿ 0 (which will be assumed valid throughout the paper) then the optimal solution xi− () of this problem
takes the following form. Let ei denote the ith unit vector and
= Z
R(Z)
−1
[I − Z
−1 t
A (AZ
−2 t −1
A)
AZ
−1
]Z
−1
:
Then
xi− () = xi −
q
xi− () = xi −
q
i
eit R(Z)e
and
i − (ct R(Z)e
i )2 =ct R(Z)c
eit R(Z)e
i ¡0
for ct R(Z)e
i ¿0:
for ct R(Z)e
Respectively, the following criterion for identifying optimal basic variables was obtained: if xi− () ¿ 0; then
the ith primal variable is an optimal basic variable, i.e., xi∗ ¿ 0 and zi∗ = 0:
We see that the smaller the size of the ellipsoid Ep ; the larger is the value of xi− (): Below we consider
the approach to diminish the size of the solution-containing ellipsoid.
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
31
3. Improved primal solution-containing ellipsoid
Denote by P ∗ the optimal solution set of (1) and let W be a closed bounded subset of Rn such that
W ⊇ P ∗ : We will refer to such a W as a primal localization. Let S ∈ Rn×n be a diagonal matrix having
positive diagonal entries si :
For any x ∈ P ∗ we have
2 − 2x t (S Z)
2x
− x)k
2 + kS Zxk
kS Z(x
2 = kS Z xk
2 − 2x t (S Z)
2 w}
6 kS Z xk
2 + max{kS Zwk
w∈W
≡ 2 (S; W );
(3)
where the inequality follows from the condition x ∈ P ∗ ⊆ W:
Then the ellipsoid
− x)k6(S;
W )}
Ep (S; W ) = {x ∈ Rn | kS Z(x
contains all optimal primal solutions.
Using the ellipsoid Ep (S; W ) the lower bound xi− can be derived similarly as in Section 2. It is not hard
for
to verify that we need only to substitute R(Z)
−1 At (A(S Z)
−2 At )−1 A(S Z)
−1 ](S Z)
−1 :
= (S Z)
−1 [I − (S Z)
R(S Z)
and use (S; W ) instead of :
The reason for using the scaled ellipsoid is as follows. Suppose for example that the pair (x;
z)
was produced
takes the form
by a primal–dual method [6]. Let S = (XZ)−1=2 : For this case R(S Z)
R(D−1 ) = D[I − DAt (AD2 At )−1 AD]D;
1=2 −1=2
: Note that the matrix AD2 At is computed and decomposed into its Cholesky factor in
where D = X Z
most of the primal–dual methods.
To dene a localization W we can use some constraints of the primal problem or a linear combination of
them. Below we consider the localization which results in a closed form expression for (S; W ):
Proposition 1. Let the primal localization be dened by W1 = {w ∈ Rn | z t w6; w¿0}: Then
2 + max{si2 ( − 2zi xi )}:
2 (S; W1 ) = kS Z xk
i
Proof. For any x ∈ P ∗ we have
t x = ct x − y t b6ct x − y t b = z t x =
z t x = (c − At y)
and hence P ∗ ⊆ W1 : Moreover since z ¿ 0 then W1 is bounded, i.e., W1 is a localization. To estimate (S; W1 )
in (3) we should solve the problem
2 − 2x t (S Z)
2 w}:
f∗ = max {kS Zwk
w∈W1
The objective function of this problem is strictly convex and hence the optimal solution is attained at a vertex
of the non-degenerate simplex W1 : The non-zero component of the ith vertex is = zi and thus
f∗ = max{si2 ( − 2zi xi )};
i
which by (3) gives the required expression for (S; W1 ):
32
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
Note that if S = I and x ¿ 0; then
2 + 2 − 2 min zi xi ¡ 2
2 (I; W1 ) = kZ xk
i
and hence Ep (I; W1 ) ⊂ Ep :
The other choice of the scaling matrix S associated with the primal–dual algorithms is S = (XZ)−1=2 : For
this case, we have
2 ((XZ)−1=2 ; W1 ) =
2
− :
mini zi xi
A similar approach can be used to derive a dual ellipsoid that contains all optimal dual slack solutions, i.e.
V )};
Ed (S; V ) = {z ∈ Rn | kS X (z − z)k6(S;
where
2 + max{kS X k2 − 2z t (S X )2 }:
2 (S; V ) = kS X zk
∈V
A dual localization V is a closed bounded subset of Rn such that z ∗ ∈ V for any optimal dual slack solution
z ∗ : It is not hard to verify that if x ¿ 0; then V1 = { ∈ Rn | x t 6; ¿0} is the dual localization and
(S; V1 ) = (S; W1 ): The ellipsoid Ed (I; V1 ) is smaller than the dual-containing ellipsoid Ed = {z ∈ Rn | kX (z −
z)k6}
considered in [1] and it can be used similarly to identify optimal non-basic primal variables.
4. Primal–dual solution-containing ellipsoid
Denote by F ∗ ⊆ R2n the set of all optimal primal–dual pairs (x∗ ; z ∗ ) and let
be a closed bounded subset
of R2n such that
⊇ F ∗ : We will refer to such an
as a primal–dual localization. In this section we assume
(x;
z)
¿ 0:
For any (x; z) ∈ F ∗ we have similar to (3):
2 + kS X k2
− x)k
2 + max {kS Zwk
2 6 2kS Z xk
kS Z(x
2 + kS X (z − z)k
(w;)∈
2 w − 2z t (S X )2 } ≡ 2 (S;
)
− 2x t (S Z)
and hence the ellipsoid
− x)k
2 + kS X (z − z)k
2 62 (S;
)}
Epd (S;
) = {(x; z) ∈ R2n | kS Z(x
contains all optimal primal–dual pairs (x∗ ; z ∗ ):
Proposition 2. Let the primal–dual localization be dened by
1 = {(w; ) ∈ R2n | z t w + xt = ; (w; )¿0}:
Then
2 + max{si2 ( − 2zi xi )}:
2 (S;
1 ) = 2kS Z xk
i
t (z − z)
= 0 and xt z = 0 by the complementarity
Proof. For any (x; z) ∈ F ∗ we have z t x + x t z = since (x − x)
∗
condition. Hence F ⊆
1 and
1 is a localization. Then the required expression for (S;
1 ) is obtained
similar to the proof of Proposition 1.
Now we use Epd to identify, if possible, the optimal basis. Consider the problem
qt x → min;
(4)
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
Ax = b;
At y + z = c;
33
(5)
− x)k
2 62 ;
kS Z(x
2 + kS X (z − z)k
(6)
z t x + x t z = ;
(7)
x t z6;
(8)
z t x6:
(9)
Here q is such that the objective of this problem coincides with xi up to a positive scale factor. In [9] the
problem similar to (4) – (6) was discussed for q = Dei :
−1 q; d−2 = ke v k2 + kek
2 − ke u k2 ; where v ; u are used to denote the projections of
Let e = S −1 e; q = (S Z)
n
−1 and AS X , respectively. Denote also
∈ R onto the null space of the matrices A(S Z)
−1
(e t qv )2
t
2
2
2
4
2
:
− ke u k ) ke v k kqv k −
B = (e qv ) (kek
ke v k2
Proposition 3. The optimal solution x− of problem (4)–(9) is such that
(a) if e t qv ¿0 and kqv k ¡ (e t qv ); then
s
2
e t qv
(e t qv )2
t −
t
2−
2−
−
k
k
q
;
q xa = q x −
v
ke v k2
ke v k2
ke v k2
2 − ke u k2 ) − 2 ]; then
(b) if e t qv 60 and 2 ¡ B[2 (kek
s
2
(e t qv )2
t −
t
2
2
−
kqv k −
;
q xb = q x −
kek
2 − ke u k2
ke v k2
(c) else
qt xc− = qt x − (e t qv )d2 −
q
(2 − 2 d2 )(kqv k2 − d2 (e t qv )2 ):
Proof. The KKT optimality conditions for (4) – (9) give
2 (x − x)
− ( − )z = 0;
q + At u + 2(S Z)
(10)
− ( − )x = 0;
v + 2(S X )2 (z − z)
(11)
Av = 0;
¿ 0;
¿0;
¿0;
where ; ; and are the Lagrange multipliers for the restrictions (7) – (9).
−2 and utilizing the condition A(x − x)
= 0 we get
Multiplying (10) by A(S Z)
−1 (( − )e − q):
−2 At ]−1 A(S Z)
u = [A(S Z)
−2 and substituting the expression for u we obtain
Multiplying (10) by (S Z)
1
−1 [( − )e − q]
v:
(S Z)
2
+ (z − z)
= 0 we get
Now multiplying (11) by A and using the conditions Av = 0; At (y − y)
x − x =
y − y = −
−
−2 At ]−1 Ax
[A(S Z)
2
34
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
and hence
=
z − z = −At (y − y)
− −1
(S X ) (e − e u ):
2
+ z t (x − x)
= −: Substituting x − x and z − z in this expression we obtain the
By (7) we have x t (z − z)
equation for which gives
2 − ke u k2 ) − 2]:
= d2 [e t qv + ke v k2 + (kek
If ¿ 0; then by the complementarity condition for (8) we have x t z=: Hence by (7) it follows 0= z t x ¡
and = 0 by the complementarity condition for (9). Similarly if ¿ 0, then = 0. We can write both these
conditions in the form = 0.
= 0 and
Remembering that = x t z the complementarity conditions for (8) and (9) take the form x t (z − z)
= 0. Substituting the expressions for x − x and z − z we obtain
z t (x − x)
( − ) = 0;
[( − )ke v k2 − e t qv ] = 0:
Utilizing the expression for and the condition = 0 we get
d2 (e t qv − ke v k2 − 2) = 0;
(12)
kek
2 − ke u k2
2
2
2
2
= 0:
−
d
(k
ek
−
k
e
k
)
−
2d
−e t qv
u
ke v k2
(13)
If e t qv 60 then the term in the brackets in (12) is strictly negative and hence = 0. By (13) we have that
if e t qv ¿0 then = 0.
Since the quadratic constraint (6) must be active, then
2
2
−
1
k( − )e v − qv k2 +
ke − e u k2 :
2 =
2
2
Substituting the expression for we obtain after the algebraic manipulations:
2
2
1
1
2
2 t
2
2
2 2
(kqv k − d (e qv ) ) −
d2 {(e t qv )( − )(kek
2 − ke u k2 )
= d +
2
2
2 − ke u k2 )]}:
+ 2[ke v k2 + (kek
(14)
Now we are in a position to derive the main results of the proposition.
Suppose that e t qv ¿0 and kqv k ¡ (e t qv ). By (13) it follows that = 0. Let ¿ 0. Then by (12) we
have
=
e t qv − 2
:
ke v k2
Substituting this in (14) with = 0 we obtain
2
2 ke v k2 − 2
1
=
:
2
kqv k2 ke v k2 − (e t qv )2
(15)
Note that since kqv k ¡ (e t qv ), then the numerator of the above expression is strictly positive. Using the
expression for it is not hard to verify that in (15) is strictly positive. Now utilizing the expressions for
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
x − x;
and we have
=
qt (x − x)
1
e t qv
1
[e t qv − kqv k2 ] = −
−
2
ke v k2
2
kqv k2 −
(e t qv )2
ke v k2
35
;
which nally proves part (a) of the proposition.
Note that if e t qv ¿0 and kqv k¿(e t qv ) then the case ¿ 0 is impossible and we have = = 0.
2 − ke u k2 ) − 2 ]. By (12) it follows that = 0. Let ¿ 0.
Suppose now that e t qv 60 and 2 ¡ B[2 (kek
Then by (13) we have
2 − ke u k2 ) + 2ke v k2
e t qv (kek
:
ke v k2 (kek
2 − ke u k2 )
Substituting this in (14) with = 0 we obtain
2
2 (kek
2 − ke u k2 ) − 2
1
ke v k2 :
=
2
(kek
2 − ke u k2 )(kqv k2 ke v k2 − (e t qv )2 )
=−
(16)
Note that under our assumption the numerator of this expression is strictly positive. Moreover, if we substitute
this in (16), then ¿ 0.
Substituting x − x;
and we have
s
2
1
(e t qv )2
t
2
t
2−
= [( − )e qv − kqv k ] = −
k
2 −
k
q
;
q (x − x)
v
2
kek
2 − ke u k2
ke v k2
which proves part (b) of the proposition.
2 − ke u k2 ) − 2 ] then the case ¿ 0 is impossible and = = 0.
Note that if e t qv 60 and 2 ¿B[2 (kek
If either (a) or (b) is not satised, then = = 0. For this case we have = d2 (e t qv − 2) and
=
qt (x − x)
d2 (e t qv − 2) t
1
1
(e t qv − kqv k2 ) =
e qv − kqv k2
2
2
2
1
(kqv k2 − d2 (e t qv )2 ):
2
Obtaining from (14) with = = 0 we get the expression for qt x− required in part (c) of the proposition.
= −d2 2 e t qv −
Since (x∗ ; z ∗ ; y∗ ) is feasible to (4) – (9) for = (S;
1 ), we have the following criterion for identifying the
optimal basis: if qt x− ¿ 0, then xi∗ ¿ 0 and zi∗ = 0.
Note that for part (c) of Proposition 3 we have = = 0 and hence constraints (8) and (9) can be
relaxed without changing the optimal value. For parts (a) and (b), either (8) or (9) is active and hence
qt xc− 6min{qt xa− ; qt xb− }.
In [9] the primal–dual containing ellipsoid centered at the origin was considered for S = (XZ)−1=2 and
(x;
z;
y)
∈ N2 (
), where
∈ [0; 1),
0
N2 (
) = (x; y; z) ∈ F | XZe − e 6
n
n
and F 0 is the set of all strictly feasible primal–dual solutions.
For (x;
z;
y)
∈ N2 (
) we have zi xi ¿(1 −
)=n; i = 1; n and by Proposition 2
2
n
2
6
≡ ;
mini zi xi 1 −
such that the size of our ellipsoid is the same as in [9]. Similarly we can estimate the size of the ellipsoid
for (x;
z;
y)
∈ N−∞ () = {(x; z; y) ∈ F 0 | zi xi ¿=n; i = 1; n}; ∈ (0; 1).
2 ((XZ)−1=2 ;
1 ) =
36
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
−
The lower bound qt xYT
obtained in [9, Theorem 3] by the solution of the problem similar to (4) – (6) for
−1
q = D ei is as follows:
q
2
t −
t
q xYT = ei P e − ( − )(1 − p ii );
where P = Dt At (AD2 At )−1 AD and p ii is the ith diagonal entry of P.
−1=2
2
−1
, part (c) of Proposition 3 gives
Since u = v = (I − P) and d = for S = (XZ)
s
1
2
−
e ]2 ¿ qt xYT
:
qt xc− = eit P e − ( − ) 1 − p ii − [eit (I − P)
Hence Proposition 3 results in the stronger criterion for identifying the optimal basis.
It is interesting to compare the lower bound obtained in Proposition 3 with the bound xi− () in Section 2.
We have
2 + 2 − 2 min zi xi = 2 + kZ xk
2 − 2 min zi xi :
2 (I;
1 ) = 2kZ xk
i
i
If (x;
z;
y)
∈ N2 (
), then (1 −
)=n6zi xi 6(1 +
)=n and
kZ xk
2 − 2 min zi xi 62 (
2 + 4
− 1)=n:
i
√
For
∈ [0; 5 − 2) the right-hand side of the latter inequality is strictly negative and (I;
1 ) ¡ .
= Z −1 (Z −1 )v for any ∈ Rn and (Z −1 c)v = ev by At y + z = c, it is not hard to verify that
Since R(Z)
xi− () coincides with the optimal solution of the following problem:
− x)k6;
Ax = b; z t x6}:
xi− () = min{xi | kZ(x
(17)
√
z;
y)
∈ N2 (
),
∈ [0; 5 − 2), = (I;
1 ) ¡ ,
Comparing (17) with the problem (4) – (9) for q = ei , S = I; (x;
we obtain xi− () ¡ eit x− that results in improved criterion for identifying the optimal basis.
The ellipsoids considered in this paper are algorithm-independent, that is they do not depend on how the
primal–dual pair (x;
z)
was obtained. Meanwhile the size of the ellipsoid depends on the proximity of (x;
z)
to
the central path: the closer to the central path, the smaller the size. This is one more reason for maintaining
(x;
z)
in the neighborhood of the central path during the course of the interior algorithm.
Acknowledgements
The author gratefully acknowledges two anonymous referees for their helpful comments and remarks.
Appendix
Since our expression for xi− () is slightly dierent from the one used in [1, Criterion (C4), p. 170], we
give here the complete derivation for xi− ().
Proposition A1. The optimal solution xi− () of the problem
2
2
t Z (x − x)6
; Ax = b; ct x6ct x}
min{eit x | (x − x)
(A.1)
is as follows:
xi− () = xi −
q
i − (ct R(Z)e
i )2 =ct PAZ −1 c
eit R(Z)e
i ¡0
for ct R(Z)e
37
I.S. Litvinchev / Operations Research Letters 27 (2000) 29–37
and
xi− ()
q
i
= xi − eit R(Z)e
i ¿0:
for ct R(Z)e
Proof. The Lagrangian, associated with (A.1) is
2
t Z (x − x)
− 2 ] + ut ( Ax − b) + (ct x − ct x):
L = eit x + [(x − x)
The KKT condition @L=@x = 0 gives
2
+ c + At u = 0:
ei + 2Z (x − x)
(A.2)
2
− ] = 0, (c x − c x)
= 0, A(x − x)
=0
Multiplying (A.2) by (x − x)
and utilizing conditions [(x − x)
Z (x − x)
we obtain
= (xi − xi )=22 :
Multiplying (A.2) by AZ
−2 t −1
u = −(AZ
A)
−2
AZ
t
2
t
t
we get
−2
(ei + c):
From (A.2) we have
−2
i + c):
2(x − x) = Z (At u + ei + c) = R(Z)(e
(A.3)
t
Multiplying (A.3) by c we obtain
i + ct R(Z)c:
062ct (x − x) = ct R(Z)e
(A.4)
If c R(Z)ei ¡ 0, then it follows from (A.4) that ¿ 0 and hence by the complementarity condition
i =ct R(Z)c.
Substituting and in (A.3) and multiplyct (x − x) = 0. From (A.4) we have = − ct R(Z)e
ing by ei we obtain the rst part of Proposition A1.
i ¿0. Suppose that we have relaxed the restriction ct x6ct x in (A.1) or, just the same, put = 0
Let ct R(Z)e
in the Lagrangian. Then similar to (A.3) we obtain for the solution of the relaxed problem
i
2(x − x) = R(Z)e
t
i ¿0 we have ct x6ct x.
and since ¿ 0 and ct R(Z)e
Since the relaxed solution is feasible to (A.1), then it is
optimal to (A.1). This immediately gives the required expression for xi− ().
Note that in [1, Criterion (C4), p. 170] there is a little
aw. The rst expression for xi− () in Proposition A1
i ¿ 0.
was associated there with the case ct R(Z)e
References
[1] I.C. Choi, D. Goldfarb, On solution-containing ellipsoids in linear programming, J. Optim. Theory Appl. 80 (1994) 161–173.
[2] J.L. Gon, J.P. Vial, On the computation of weighted analytic centers and dual ellipsoids with the projective algorithm, Math.
Programming 60 (1993) 81–92.
[3] M. Kojima, Determining basic variables of optimal solutions in Karmarkar’s new LP algorithm, Algorithmica 1 (1986) 499–515.
[4] G. Sonnevend, An ‘Analytical Centre’ for Polyhedrons and New Classes of Global Algorithms for Linear (Smooth, Convex)
Programming, in: Lecture Notes in Control and Information Sciences, Vol. 84, A. Prekopa, J. Szelezsan and B. Strazicky (Eds.),
Springer, Berlin, 1986, pp. 866 –875.
[5] M.J. Todd, Improved bounds and containing ellipsoids in Karmarkar’s linear programming algorithm, Maths. Oper. Res. 13 (1988)
650–659.
[6] S.J. Wright, Primal–Dual Interior-Point Methods, SIAM Publications, Philadelphia, PA, 1997.
[7] Y. Ye, Recovering the optimal basis in Karmarkar’s polynomial algorithm for linear programming, Math. Oper. Res. 15 (1990)
564–572.
[8] Y. Ye, A “build-down” scheme for linear programming, Math. Programming 46 (1990) 61–72.
[9] Y. Ye, M.J. Todd, Containing and shrinking ellipsoids in the path-following algorithm, Math. Programming 47 (1990) 1–9.