Directory UMM :Data Elmu:jurnal:O:Operations Research Letters:Vol26.Issue2.Mar2000:

Operations Research Letters 26 (2000) 49–54
www.elsevier.com/locate/orms

Uniform bounds on the limiting and marginal derivatives of the
analytic center solution over a set of normalized weights
A.G. Holder a; ∗ , R.J. Caronb
b Department

a Department of Mathematics, Trinity University, 715 Stadium Drive, San Antonio, TX 78212-7200, USA
of Economics, Mathematics and Statistics, University of Windsor, 401 Sunset Avenue, Windsor, Ont., Canada N9B 3P4

Received 1 June 1998; received in revised form 1 September 1999

Abstract
While properties of the weighted central path for linear and non-linear programs have been studied for years, the eect that
weighting the central path has on its limiting derivatives has not been fully explored. We show that the limiting derivatives
of the central path for a linear program are uniformly bounded over a set of normalized weights. Consequently, this allows
a similar bound for the marginal derivatives of the analytic center solution with respect to changes in the right-hand side.
c 2000 Elsevier Science B.V. All rights reserved.

Keywords: Analytic central path; Computational economics; Sensitivity analysis

1. Introduction
Consider the standard form primal linear programming problem
(LP)

min{cx: Ax = b; x ¿ 0}

and its associated dual:
(LD) max{yb: yA + s = c; s ¿ 0};
where A ∈ Rm×n ; m 6 n, rank(A) = m; b ∈ Rm ; c ∈
Rn ; x ∈ Rn ; s ∈ Rn ; y ∈ Rm , and c; y, and s are
taken to be row vectors. Since linear changes in the
right-hand side, b, are considered, the primal and dual
feasible regions are denoted by Pb , and D, respectively. Similarly, the primal and dual optimal faces
are Pb∗ and Db∗ , respectively.
∗

Corresponding author.

Both the primal and dual feasibility regions are assumed to satisfy Slater’s constraint qualication. That

is, we assume the existence of an x ∈ Pb and a pair
(y; s) ∈ D, such that x ¿ 0 and s ¿ 0. The right-hand
side vector, b, is admissible if the corresponding LP
satises Slater’s constraint qualication.
The assumption that b is admissible implies the existence of the omega central path [15],
{(x(; b; !); (y(; b; !); s(; b; !))):  ¿ 0};
where ! ∈ Rn++ and the elements are the unique solution to
Ax(; b; !) = b;
y(; b; !)A + s(; b; !) = c;
S(; b; !)x(; b; !) = !:

c 2000 Elsevier Science B.V. All rights reserved.
0167-6377/00/$ - see front matter
PII: S 0 1 6 7 - 6 3 7 7 ( 9 9 ) 0 0 0 7 3 - 5

50

A.G. Holder, R.J. Caron / Operations Research Letters 26 (2000) 49–54

In the last equality, S(; b; !) is the diagonal matrix

formed by s(; b; !). These equations are the necessary and sucient Lagrange conditions for
)
(
n
X
!i ln(xi ): Ax = b; x ¿ 0
min cx − 
i=1

and
(

max yb + 

n
X

)

!i ln(si ): yA + s = c; s ¿ 0 :

i=1

Furthermore,
lim (x(; b; !); (y(; b; !); s(; b; !)))

 → 0+

Furthermore, we use D+ x∗ (b; !) to represent
lim → 0+ D x(; b; !). The existence of the rst-order
limiting derivatives was shown by Witzgall et al. [20]
and Adler and Monteiro [1]. Guler [13] established
the existence of the higher-order limiting derivatives.
This result was used in [9] to show that along any
given direction b, the analytic center solution is
one-sided innitely, continuously dierentiable.
In Section 2, the vector of right-sided derivatives
of x∗ (b; !) with respect to , is shown to be uniformly bounded over {! ∈ Rn++ : k!k = 1}. In Section 3, a similar result is presented for the right-sided
marginal derivatives, denoted D+ x∗ (b ; !), where
b = b + pb. Notice that for any admissible b and

any b; b is admissible for suciently small .

= (x∗ (b; !); (y∗ (b; !); s∗ (b; !)))
2. The limiting derivatives of the omega central path
is a strictly complementary solution to the linear programs, called the omega analytic center solution. This
solution induces the optimal partition:
B(b) = {i: xi∗ (b; !) ¿ 0}
and
N (b) = {1; 2; 3; : : : ; n} \ B(b):
These sets do not depend on any specic choice of
! ∈ Rn++ . If no confusion results, the argument (b)
is dropped. The sets B and N are used as subscripts
to indicate the components of a vector, or columns
of a matrix, that correspond to elements in B and N ,
respectively.
Sonnevend [16 –18] showed the mathematical programming community that analytic centers were important because of their connection to interior point
algorithms. In fact, a polynomial-time algorithm for
linear programming problems seems to require a centering component in its search direction [8]. Because
of this, properties of the omega central path are important and have been investigated by many researchers
[3,4,6,10 –12,14,21,22].

In this paper, the limiting derivatives of the ! central path are of interest. For any  ¿ 0, the derivative
of x(;
 b; !) is denoted by
x(;
 b; !) − x(; b; !)
:
 b; !) = lim
D x(;
 → 
 − 

The components of D+ x∗ (b; !) are partitioned into
D+ xB∗ (b; !) and D+ xN∗ (b; !). The next lemma shows
that D+ xN∗ (b; !) is uniformly bounded over a set of
normalized weights.
Lemma 1. For any admissible b; there exists a constant M ¿ 0 such that
sup{kD+ xN∗ (b; !)k: ! ∈ Rn++ ; k!k = 1} 6 M ¡ ∞:
Proof In all of [1,13,20], it is shown that
D+ xN∗ (b; !) = (SN∗ (b; !))−1 !N :

(1)

Consider,
(
X
!i ln(si ): yAB = cB ;
max
i∈N

)

yAN + sN = cN ; sN ¿ 0 ;
with solution (y∗ (b; !); sN∗ (b; !)). The necessary and
sucient optimality conditions for this program are
the existence of u∗ (b; !) such that
u∗ (b; !) ¿ 0;

(2)

Au∗ (b; !) = 0;

(3)

y∗ (b; !)AB = cB ;

(4)

y∗ (b; !)AN + sN∗ (b; !) = cN ;

(5)

A.G. Holder, R.J. Caron / Operations Research Letters 26 (2000) 49–54

SN∗ (b; !)uN∗ (b; !) = !N ;

(6)

sN∗ (b; !) ¿ 0:

(7)

∗

∗

Let (u;
ˆ (y;
ˆ sˆN )) = (u (b; e); (y (b; e);
where e is the vector of ones. Then,

sN∗ (b;

e))),

uˆ N ¿ 0;

(8)

Auˆ = 0;

(9)

yA
ˆ B = cB ;

(10)

yA
ˆ N + sˆN = cN ;

(11)

sˆN ¿ 0:

(12)

From (2), (7), (8), and (12) it follows that for all
i ∈ N,
sˆi ui∗ (b; !) 6 sˆN uN∗ (b; !)
∗

+

sN∗ (b; !)uˆ N :

(13)

From (3) and (9), u (b; !) − uˆ ∈ null(A), and from
(5) and (11), (0; sN∗ (b; !)) − (0; sˆN ) ∈ row(A). Since
null(A)⊥ = row(A),
sˆN uN∗ (b; !) + sN∗ (b; !)uˆ N
= sN∗ (b; !)uN∗ (b; !) + sˆN uˆ N :

(14)

Combining (6), (13), and (14) with the fact that ! is
normalized, gives
sˆi ui∗ (b; !) 6 eTN !N

+ sˆN uˆ N 6 n + sˆN uˆ N :

What is left to show is that D+ xB∗ (b; !) is bounded.
From Guler [13, Theorem 2:1] and Eq. (1),
D+ xB∗ (b; !)
= −
B−1=2 XB∗ (b; !)(AB
B−1=2 XB∗ (b; !))+
× AN (SN∗ (b; !))−1 !N
= −
B−1=2 XB∗ (b; !)(AB
B−1=2 XB∗ (b; !))+
× AN (D+ xN∗ (b; !));

Lemma 2 (Stewart [19]). Let M ∈ Rp×q . Then; there
is a number M ¿ 0; such that
sup kPD k 6 M:
D ∈ D++

Proof Let
C = {u ∈ col(M ): kuk = 1}
and
[

K=

leftnull(DM ):

D ∈ D++

We show that K ∩ C = ∅, so that
inf

(u;v) ∈ C×K

kv − uk ¿ 0;

and then use this bound to complete the result.
Suppose for the sake of attaining a contradiction
that K ∩ C 6= ∅. Then, there exists a sequence
{v k ∈ K} → u ∈ C. For every k there exists D k ∈ D++
such that v k ∈ leftnull(D k M ) and u ∈ col(M ). So,
and
(D k )1=2 u ∈
(D k )1=2 v k ∈ leftnull((D k )1=2 M )
k 1=2
col((D ) M ). Hence,
X
ui dki vi :
0 = vkD ku =
ui 6= 0

(16)

where the superscript “+” denotes the Moore–Penrose
generalized inverse. Hence, Lemma 1 implies that to
bound D+ xB∗ (b; !), only

B−1=2 XB∗ (b; !)(AB
B−1=2 XB∗ (b; !))+

corollary. The absence of a rank requirement makes
the statements of Lemma 2 and Corollary 1 slight
generalizations of Theorem 1 in [19] due to Stewart.
Stewart’s proof, without modication, continues to
work for the relaxed statement found in the next
lemma. However, the proof of the corollary does not
follow from the proof found in [19] without an additional property of the Moore–Penrose generalized
inverse. Dene D++ to be the set of positive diagonal matrices. For any D ∈ D++ , set MD = D 1=2 M
and PD = MMD+ D 1=2 . The following notation is used
in the proof of Lemma 2: col(M ) = {Mx: x ∈ Rn },
leftnull(M ) = {v: vM = 0}, and K is the closure of
the set K.

(15)

The following completes the result and holds from
(6), (7) and (15),
n + sˆN uˆ N
!i
= ui∗ (b; !) 6
:
si∗ (b; !)
sˆi

51

(17)

needs to be bounded. Lemma 3 provides the needed
bound, and depends on Lemma 2 and its subsequent

Since kuk = 1, some ui 6= 0. However, for suciently
large k, ui vik ¿ 0 when ui 6= 0, which implies the contradiction that the right-hand side of the last equality
is positive. Hence, K ∩ C = ∅. Dene
≡

inf

(u;v) ∈ C×K

kv − uk ¿ 0:

For any xed D ∈ D++ ; kPD k = maxkwk = 1 kPD wk.
Let kwk = 1 and w = u + v where u ∈ col(M ) and

52

A.G. Holder, R.J. Caron / Operations Research Letters 26 (2000) 49–54

v ∈ leftnull(DM ). Since D 1=2 PD x is the projection of
D 1=2 x onto col(D 1=2 M ), we have
D

1=2

PD w = D

1=2

PD u + D

=D

1=2

PD u:

1=2

PD v

1=2

Proof Let D =
B−1=2 XB∗ (b; !) and M T = AB . Then,
k
B−1=2 XB∗ (b; !)(AB
B−1=2 XB∗ (b; !))+ k
= kD(M T D)+ k = k(DM )+ Dk:

1=2

1=2

Furthermore, because D u ∈ col(D M ), D PD u =
D 1=2 u. Hence PD w = u, and to show that kPD k is
bounded we may show that kuk is bounded. Without loss of generality we assume that u 6= 0. Since
(u=kuk) ∈ C and (v=kuk) ∈ leftnull(DM ),
kwk
1
=
kuk
kuk
ku + vk
=
kuk


u
v


+
=
kuk kuk
6 :

From Corollary 1 there exists a constant M ¿ 0 such
that for all ! ∈ Rn++ ,
k
B−1=2 XB∗ (b; !)(AB
B−1=2 XB∗ (b; !))+ k
6 kA+
B kM ¡ ∞:
The following theorem is the main result of this
section, and it shows that the limiting derivatives of the
omega analytic center solution are uniformly bounded
over a set of normalized weights.
Theorem 1. For any admissible right-hand side b;
there exists a constant M ¿ 0 such that
sup {kD+ x∗ (b; !)k: k!k = 1; ! ¿ 0} 6 M ¡ ∞:

So, kPD k 6 1=, and since  is independent of D, we
have that
1
sup kPD k 6 :

D ∈ D++
Corollary 1. There exists M such that
sup k(D 1=2 M )+ D 1=2 k 6 kM + kM:
D ∈ D++

Proof Partition the vector of derivatives to obtain
kD+ x∗ (b; !)k 6 kD+ xB∗ (b; !)k + kD+ xN∗ (b; !)k:
Lemma 1, Eq. (16), and Lemma 3, imply the existence
of an M1 and M2 such that
sup {kD+ x∗ (b; !)k: ! ∈ Rn++ ; k!k = 1}
6 kATB kM1 kAN kM2 + M2
6 M ¡ ∞:

Proof From Corollary 1:4:1 in [5],
3. The marginal derivatives of the omega central
path

(D 1=2 M )+ = M + (D 1=2 MM + )+ :
Hence,
M + PD = M + M (D 1=2 M )+ D 1=2
+

+

+

1=2

= M MM (D
= M (D

1=2

+ +

MM ) D

1=2

+ +

MM ) D 1=2 = (D 1=2 M )+ D 1=2 :

So,
sup k(D 1=2 M )+ D 1=2 k =
D ∈ D++

sup kM + PD k
D ∈ D++

6 kM + k sup kPD k
D ∈ D++
+

6 kM kM;
which completes the result.
Lemma 3. The matrix in (17) is uniformly bounded
over {! ∈ Rn++ }.

In this section, the boundedness of the rst-order
limiting derivatives is shown to imply that the
rst-order marginal derivatives are also bounded.
This follows because D x∗ (b ; !) is expressible as
a limiting derivative of an omega central path. The
following subset relationships are needed.
Lemma 4 (Adler and Monteiro [2]). For a given b
and suciently small ,
B(b) ⊆ B(b )

and N (b) ⊇ N (b ):

Lemma 4 shows that if b is a direction of change
that forces an immediate change in the optimal partition, then the cardinality of B increases. To accommodate such a situation, dene for suciently small

A.G. Holder, R.J. Caron / Operations Research Letters 26 (2000) 49–54

53

;
B = B(b )=B(b). From [7] we know that B(b ) is
invariant on (0; ),
 for  suciently small. The establishment of the fact that the marginal derivatives
of x∗ (b; !) are the limiting derivatives of an omega
central path is found in Lemma 5. The main result
follows.

also wishes to thank Andrew Knyazev. The work of
the second author is supported by the Natural Sciences
and Engineering Research Council of Canada.

Lemma 5 (Holder et al. [9]). For a given b; let

[1] I. Adler, R. Monteiro, Limiting behavior of the ane scaling
continuous trajectories for linear programming problems,
Math. Programming 50 (1991) 29–51.
[2] I. Adler, R. Monteiro, A geometric veiw of parametric linear
programming, Algorithmica 8 (1992) 161–176.
[3] D. Bayer, J. Lagarias, The nonlinear geometry of linear
programming. I, Trans. Amer. Math. Soc. 314 (2) (1989)
499–526.
[4] D. Bayer, J. Lagarias, The nonlinear geometry of linear
programming. II, Trans. Amer. Math. Soc. 314 (2) (1989)
527–581.
[5] S. Campbell, C. Meyer Jr., Generalized Inverses of Linear
Transformations, Fearon Pitman Publishers, Belmont, CA,
1979.
[6] J. Ding, T.Y. Li, An algorithm based on weighted logarithmic
barrier functions for linear complementarity problems,
Arabian J. Sci. Eng. 15 (4) (1990) 679–685.
[7] H. Greenberg, Rim sensitivity analysis from an interior
solution, Technical Report CCM No. 86, University of
Colorado, Denver, 1996.
[8] D. Den Hertog, C. Roos, A survey of search directions
in interior point methods for linear programming, Math.
Programming 52 (2) (1991) 481–510.
[9] A. Holder J. Sturm, S. Zhang, The analytic central path,
sensitivity analysis, and parametric programming, Technical
Report CCM No. 118, Center for Computational Mathematics,
The University of Colorado, Denver, 1997.
[10] L. McLinden, An analogue of Moreau’s proximation theorem,
with applications to the nonlinear complementary problem,
Pacic J. Math. 88 (1) (1980) 101–161.
[11] N. Megiddo, Pathways to the optimal set in linear
programming, in: N. Megiddo (Ed.), Progress in
Mathematical Programming: Interior-Point Algorithms and
Related Methods, Springer, New York, 1989, pp. 131–158.
[12] S. Mizuno, M. Todd, Y. Ye, A surface of analytic centers
and primal-dual infeasible-interior point algorithms for linear
programming, Math. Oper. Res. 20 (1) (1995) 135–162.
[13] O. Guler, Limiting behavior of weighted central paths
in linear programming, Math. Programming 65 (1994)
347–363.
[14] C. Roos, D. Hertog, A polynomial method of approximate
weighted centers for linear programming, Technical Report
89-13, Delft University of Technology, Delft, Netherlands,
1989.
[15] C. Roos, T. Terlaky, J.-Ph. Vial, Theory and Algorithms
for Linear Optimization: An Interior Point Approach, Wiley,
New York, 1997.
[16] G. Sonnevend, An “analytic centre” for polyhedrons and
new classes of global algorithms for linear (smooth, convex)

{(zB (); z
B (); ()):  ¿ 0}

(18)

be the ! central path for
min {: AB zB + A
B z
B − b = b;
zB ¿ 0; z
B ¿ 0;

 ¿ 0}:

Then;
D+ xB∗ (b0 ; !) = D+ zB∗
and
∗
∗
D+ x
B
(b0 ; !) = D+ z
B
:

Theorem 2. For any b; there exists a constant M
with
sup {kD+ x∗ (b0 ; !)k: k!k = 1; ! ¿ 0} 6 M ¡ ∞:
Proof Let v be the dual slack vector associated with
the omega central path in (18). Since the optimal partition for (18) is (B|
B ∪ {}), Eq. (16) and Lemma
4 imply that
D+ xB∗ (b0 ; !) = D+ zB∗ (b; !)
= −
B−1=2 ZB∗ (b; !)(AB
B−1=2 ZB∗
 ∗

x
B (b; !)
+
:
× (b; !)) AN D+
∗
Since,
 ∗

x
B (b; !)
∗
= (V{
B∪{}}
(b; !))−1 !{
B∪{}} ;
D +
∗

∗
(b0 ; !) is uniformly bounded
the result that D+ x{B∪B}
over {! ¿ 0: k!k = 1} is analogous to Theorem 1.
Furthermore, since xN∗ \
B (b ; !) = 0 for suciently
small ; D+ xN∗ \
B = 0, and the proof is complete.

Acknowledgements
The authors thank Harvey Greenberg for his comments on early drafts of this article. The rst author

References

54

A.G. Holder, R.J. Caron / Operations Research Letters 26 (2000) 49–54

programming, in: A. Prekopa, J. Szelezsan, B. Strazicky
(Eds.), Lecture Notes in Control and Information Sciences,
Vol. 84, Springer, Heidelberg, 1986, pp. 866 –875.
[17] G. Sonnevend, An implementation of the method of analytic
centers, in: A. Bensoussan, J. Lions (Eds.), Lecture Notes
in Control and Information Sciences, Vol. 111, Springer,
Heidelberg, 1988, pp. 297–308.
[18] G. Sonnevend, New algorithms in convex programming based
on a notion of “centre” (for Systems of Analytic Inequalities)
and on rational extrapolation, in: K. Homan et al, (Eds.),
International Series of Numerical Mathematics, Vol. 84,
Birkhauser, Basel, 1988, pp. 311–326.

[19] B. Stewart, On scaled projections and pseudoinverses, Linear
Algebra Appl. 112 (1989) 189–193.
[20] C. Witzgall, P. Boggs, P. Domich, On the convergence
behavior of trajectories for linear programming, Contemporary Math. 114 (1990) 161–187.
[21] G. Zhao, On the relationship between the curvature integral
and the complexity of path-following methods in linear
programming, SIAM J. Optim. 6 (1) (1986) 57–73.
[22] G. Zhao, J. Zhu, Analytic properties of the central trajectory
in interior point methods, in: D. Du, J. Sun (Eds.), Advances
in Optimization and Approximation, Kluwer Academic
Publishers, Netherlands, 1994, pp. 362–375.