THE CHOLESKY UPDATE FOR UNCONSTRAINED OPTIMIZATION

40

KomuniTi, Vol.IV No.1 Januari 2012

THE CHOLESKY UPDATE FOR UNCONSTRAINED OPTIMIZATION

Saad Shakir Mahmood
College Of Education, Department Of Mathematics
Almustansiriya University
Email: saadshakirmahmood@yahoo.com

ABSTRACT
It is well known that the unconstrained Optimization often arises in economies, finance,
trade, law, meteorology, medicine, biology, chemistry, engineering, physics, education, history,
sociology, psychology, and so on. The classical Unconstrained Optimization is based on the
Updating of Hessian matrix and computed of its inverse which make the solution is very
expensive. In this work we will updating the LU factors of the Hessian matrix so we don’t need to
compute the inverse of Hessian matrix, so called the Cholesky Update for unconstrained
optimization. We introduce the convergent of the update and report our findings on several
standard problems, and make a comparison on its performance with the well-accepted BFGS
update.

Keywords : Unconstrained Optimization, Cholesky Factorization, Convergence
ABSTRAK
Diketahui bahwa masalah optimasi tak berkendala mempunyai peran pada ilmu ekonomi,
keuangan, Hukum Metrologi, Kedokteran, Biology, Kimia, Teknik, Fisika, pendidikan, Sejarah,
Sosial, Psikologi , dan lain lainnya. Optimasi tak berkendala yang klasik selama ini
menyelasaikan masalah berdasarkan penyesuaian matrik HJessian dan menghitung kebalikan dari
matrik Hessian yang membuat beban komputasi sangat mahal. Pada naskah ini, kami akan
menyesuaikan LU factor dari matrik Hessian Untuk itu kami tidak perlu menghitung kebalikan
dari matrik Hessian dank arena itu penyesuaian ini kmami namakan Penyesuaian Cholesky untuk
masalah optimasi tak berkendala. Kami memperlihatkan kekonvergenan pe.nyesuaian tersebut
dan melaporkan penumuan kami pada beberapa masalah standar yang kami selesaikan dengan

KomuniTi, Vol.IV No.1 Januari 2012

41

menggunakan penyesuan tersebut, kemudian kami memandingkan hasil yang kami dapat dengan
hasil dari penyesuaian BFGS yang terkenal pada masalah potimasi tak berkendala.
Kata Kunci : Unconstrained Optimization, Cholesky Factorization, Convergence
INTRODUCTION

Given f : R n → R , it is assumed that:

LEMMA

1.

f is twice continuously differentiable.

If β ∈ R and a ∈ R" such that a T a ≠ 0 , then

2.

f is uniformly convex, i.e., there are m1

the minimum value of x x = x 2 , such that

m2 > 0

and
2

m1 a

such

that

2

≤ a ∇ 2 f a ≤ m2 a ,
T

T

T
a x = β is x = β

were

THE CHOLESKY UPDATE

update the Jacobian matrix of f on the basis
of both L' unit lower triangular and U '
triangular

Jacobian

.

condition but it’s not symmetric.

The method proposed by Hart (1990) is to

B = L'U '

a a

The update satisfies the Quasi-Newton

a ∈ R" .

upper

a
T

matrices.

He

takes

the current approximation of
matrix

Given f : R" → R , suppose that B is
the current approximation of the Hessian
matrix, and B + is the next approximation of
the Hessian matrix. Suppose also that B is

and

he

defines

with

the

updating

and U '∈ R n×n , with L' unit lower triangular

matrices O and Q lower triangular and

matrix and U ' upper triangular matrix, such

upper triangular, respectively. Then he

that

B + = ( L'+O)(U '+Q) ,

positive definite. There exists matrices L'

determines O and Q from the following

B = L'U '

relations:

then L+ and U + are to be updated,

( L '+ O ) r = y
(U '+ Q ) s = r

and he used the following elementary lemma
to solve the above system:

(1)

such that

L+ = L'+O

U + = U '+Q
with O and Q are

(2)
(3)
lower triangular and

upper triangular matrices respectively, and

42

KomuniTi, Vol.IV No.1 Januari 2012

B + = L+U +

(4)

T

Since B is symmetric then there is a
diagonal matrix D such that
T

U '= DL'

(10)

follows:
1
2

B = L' D D L'T
 d11 0


0
d 22
1

2
D = .
.
.
.

0 0 .


with

s Bs

LT s .

obtained by using the elementary lemma as

T

1
2

T

The solution of system (8) is

(5)

B = L' DL'

and

s y

r=

l ij+ =

. . 0

. . .

. . .
. . 0

.
d nn 


yi r j

, i = 1,

n

∑r
k =i

n − 1 and j = 1,

n −1

rj − ∑ lkj* sk
lnj+ =

k= j

sn

, j = 1,

,n

(12)

It is clear that the update is symmetric. In

Hence

order to show that the update satisfies the
B=LL

T

(6)

1
2

with L = L' D , and O = QT so (4) becomes

B + = L+ L+

T

Quasi-Newton

condition,

the

lemma is introduced:

Lemma 1

The Cholesky update given in (11) and (12)

L+ r = y

satisfies the Quasi-Newton condition.

+T

L s=r

(8)

Proof:
n

with r ∈ R n , from (8)

It is only needed to show that

∑l
j =1

r r = s L+ L+ s = s B + s = s y
T

T

T

T

(9)

so the best choice of r which guarantees

.

convergence is
n

∑l
j =1

+
nj j

following

(7)

by the Quasi-Newton condition it follows

T

i (11)

2
k

r = ln+1r1 + ln+2 r2 +

+
rn
+ lnn+ −1rn −1 + lnn

n −1
n −1




+
+
r
l
s
r
−
−
 2 ∑ lk 1 sk 
 1 ∑ k1 k 
k =2
k =1
+
 + r2 
= r1 
s
sn




n





r − l+ s  r2
+ rn −1  n −1 n −11 n −1  + n
sn

 sn

+
nj j

r = yn

KomuniTi, Vol.IV No.1 Januari 2012

=

− y1 s1 − y 2 s 2 −

− y n −1 s n −1 + y s
T

sn

THE

CONVERGENCE

=

yn sn
= yn
sn

OF

THE

UPDATE

i.

Mx 2 ≤ M

F

x 2;

ii.

M

F

≤c M

2

for some c > 0 ;

iii.

M

F

= LL (M ) F

Assumption I
(a) ∇f : R" → R"

is

43

continuously

differentiable in an open convex set D ,
(b) there is an x* such that ∇f ( x * ) = 0 and

Definition 2

Let φ : R n × M L → M L , and

∇ 2 f ( x ) is nonsingular,
*

(c) there is a constant K and p > 0 such
that

∇ 2 f ( x) − ∇ 2 f ( x ) ≤ K x − x
*

* p

( x , L* ) be an element of R"×M L . The
*

update

φ

function

,

In proving a convergence result of the same
nature as for other Newton-like methods, a

Let

has

the

(

)

bounded

deterioration property at x* , L* if for some

(

)

neighborhood D1 × D2 of x* , L* there exist

α1

and

α2

such

that

if

Lipschitz condition is assumed on the

constants

Hessian matrix. Using factorization and an

x ∈ D1 , LLT s = −∇f ( x) and x + = x + s , then

open pivoting strategy, the factorizations

for any L+ ∈ φ ( x, L ) .

[

must be shown to be continuous. This has
been established in Dennis and Schnabel
(1983).
Definition 1:

Let

{

( )

M L = M ∈ L R n : mij = 0,1 < j

}

+

]

+

L+ − L * ≤ 1 + α 1σ ( x, x ) L − L * + α 2σ ( x, x )
p
p
+
+
where σ ( x, x ) = max  x − x * , x − x * 



.

lower triangular matrices. Furthermore let

Assumption II

the following mapping be defined:

Let ∇ 2 f ( x*) be LLT - factorable, i.e.

( )

LL : L R n → M L

∇ 2 f ( x ) = L * L *T ,
*

where

L* ∈ M L ,

is

such that LL (M ) = M .

invertible, and ∇f ( x*) = 0 .

Lemma 2 (Royden 1968)

Lemma 3 (Technical Lemma, Hart, 1990)

44

KomuniTi, Vol.IV No.1 Januari 2012

Let L ∈ M L , and δ , γ > 0 satisfy
(i)
(ii)

(a)

L − L * ≤ 2δ < 1 ,

{

L+ ∈ φ ( x.L ) ,

(iii)
then

}

the

{x k } ,

sequence

defined

by

max L * , L *−1 , ∇f ( x*) ≤ γ .

x k +1 = x k + s k , LLT s = −∇f ( x) satisfies for

Then each of the following holds:

each

L is invertible and L−1 ≤

γ
1 − 2γδ

L , L−1

Moreover, each of

is uniformly

bounded.

(b)

L ≤ 2δ + γ ,

(c)

L−1LT − L *−1 L *T ≤ 2δγ

(d)

LL − L * L * ≤ 4δ (2δ + γ ) .
T

,

x k +1 − x * ≤ p ⋅ x k − x * ;

k ≥ 0,

1+ γ 2
,
1 − 2δγ

T

Proof

p ∈ (0,1) .

Let

Choose

a

neighborhood D1 × D2 guaranteed by the

Note that, this technical lemma involves

bounded

some useful norm inequalities. Since the

restricted so that it contains only invertible

lemma (and the proof) is independent of the

matrices. Given c in lemma 2, and γ as in

way the lower triangular L is used, it is the

lemma 3, choose ε ( p) and δ ( p) such that

choice to use L within the context of the

( x, L) ∈ D1 × D2 whenever x0 − x * < ε and

Cholesky update. The inequality (c) is thus

deterioration

property,

of no special interest. However, (d) is

L − L * ≤ 2δ , such that

needed to represent the addition to the

2δγ < 1

original lemma. The proof for (d) is

r2
Kε p + 4δ (δ + γ ) ≤ p
2
(1 − 2γδ )

straightforward.

Theorem 1
Let f satisfy assumption I, and assume that
the update function φ satisfies the bounded
deterioration at ( x*,L *) . If for any p ∈ [0,1] ,
there exists an ε = ε (r ) and δ = δ (r ) such

[

]

p
(2α1δ + α 2 ) ε p ≤ δ

1− r

(13)

where α1 and α 2 are determined by D2 and
the bounded deterioration property. First, it
is to be noted that
x 1 − x* = x 0 − x * − LT0 L−01∇f ( x 0 )
−1

= − LT0 L−01 [∇f ( x 0 ) − ∇f ( x *) −
−1

∇ 2 f ( x *)( x 0 − x*) −

that

and

(i)

x0 − x * < ε ,

( L0 LT0 − ∇ 2 f ( x*))( x 0 − x*)]

(ii)

L0 − L * ≤ δ ,

Consequently, by the above lemmas,

KomuniTi, Vol.IV No.1 Januari 2012

+

x = x − ( LLT ) −1 ∇ f ( x )

x1 − x * ≤ LT0 L10 [Kσ ( x 0 , x*) + 4δ (2δ + α )] x 0 − x *
−1

≤

[

]

r2
Kε p + 4δ (2δ + γ ) x 0 − x *
(1 − 2γδ ) 2

must

45

satisfy

+
x − x* ≤ x − x* .

Lemma 4 (Bartle 1975):

{xn }

Let

≤ ρ x0 − x *

be a sequence of positive real

numbers. Then

∑x

n

converges if and only

n

if the sequence S = {sn } of partial sums is
Thus x1 − x * ≤ ε and x1 ∈ D1 . By way of
induction.

Assume

that

bounded.

for

, m − 1 , then L − L * ≤ 2δ , and

k = 0,1,2,

Lemma 5 (Bartle 1975):

If

x k +1 − x * ≤ ρ x k − x * .

∑x

n

converges, then lim x k = 0 .
k →∞

n

Then by the bounded deterioration

Now in this direction the following result

property and the fact that

will be needed. The straightforward proof is

p
p
α (x k +1 , x k ) = max  x k +1 − x * , x k − x *  ≤ ρ pk ε p ,

omitted.





it following

Lk +1 − L * − L − L * ≤ 2α1δρ pk ε p + α 2 ρ pk ε p .
summing both sides from k = 0 to m − 1 , a

Corollary 1 (Dennis and Schnabel, 1983)

If f satisfies the assumptions I and

telescoping sum on the left is obtained and
II

thus
Lm − L * ≤ L0 − L * + ( 2α1δ + α 2 )ε

Hence,

by the

technical

p

m −1

∑ρ

pk

k =0

lemma,

and

Lm ≤ 2δ + γ . It follows analogously to the
case m = 1 that

Observe that for any ε > 0 , there is a δ > 0
such that if x − x * < ε and L − L * ≤ 2δ
the

computed

for

any

ε > 0,

there

is

a

neighborhood Dε of ( x*, L*) such that if

( x, L) ∈ Dε , then σ ( x , x + ) < ε .
Corollary 2

Define η = y − L r , with r computed
from the Cholesky update which satisfies

x m +1 − x * ≤ ρ x m − x *

then

≤ 2δ

then

iterate

L+ r = y . Then η = T y

with T

triangular, bounded, and T * = ( L*)−1 .

Proof

lower

46

KomuniTi, Vol.IV No.1 Januari 2012

That η = T y and T lower triangular

and from (14)

are obvious. The bounded- ness comes from

i

∑r

vi =

k =1

the fact that it is invertible and L satisfies
the hypothesis of

theorem 1. The fact

that T * = ( L*)−1

is obtained by direct

to

obtain

1

sn2
Mε

on

r3 . . . rn ]

T

≤

1
p

If we set ρ1 =

In particular, v1 must

is required.

thus by
,

1
,
vi

≤

1
p

Mε

update,

bound

, n, v1 = [r1 r2

k=

and

so

bounded

deterioration property for the

v1 , i = 1,2,3,

T

1

thus
order

= y s

2
k

vi ≤ M s ≤ M ε p

Lemma 6,

substitution.
In

n

∑r

≤

2
k

approach to zero at essentially the same rate

ρ1 ≤

s

≤

2
p

Mε
sn2

s
.
vi

then

1

Mε p

s
vi

(15)

as s .
Lemma 6 (Byrd and Nocedal, 1989).

i

∑r

r12 ≤

Since

k =1

2
k

,

then

Under assumption I, it follows
T

y

m s ≤ y s ≤ s M,
2

2

T

2

T

y s

≤ M , for m and M ∈ R.

y s

r =
2
1

T

n

∑l

s B s k =1

s , by Theorem 1 L is

kl k

1

boded, and hypothesis s ≤ ε p , so there is a
number ξ ij > 0 such that lij ≤ ξ ij for all i and

Lemma 7

Let f satisfies the assumption I. Let vi
defined as vi = [r1

r2 . . . ri ]

T

There

exist numbers ε , ρ1 and ρ 2 such that if
+

σ ( x , x ) < ε , then

ρ1 ≤

, n.

n

T

s Bs

n

∑∑ ξ ki ε
i =1 k =1

1
p

≤ vi

By lemma 6, and the boundedness property

that

Proof

ms

ξ

n

n

∑∑ ξ
i =1 k =1

1
p

ki

ε ≤ vi

Since sn ≤ s , then it follows

By the Cholesky update
T

j , and such that

of L and s , exist a number ζ > 0 such

s
≤ ρ 2 , i = 1,2,3,
vi

r r = r12 + r22 +

T

y s

+ rn2 = y s
T

(14)

ms n2

ξ

n

n

∑∑ ξ
i =1 k =1

1
p

ki

ε ≤ vi

KomuniTi, Vol.IV No.1 Januari 2012

so,

s
vi

ζε

≤

n

Let f satisfies assumption I and II. Let α *

1
p

n

ms n2 ∑∑ ξ ki ε

be as above. Then there exist numbers

1
p

Lemma

ζε

ρ2 =

thus

ms

holds

by

letting

1
p

n

2
n

n

∑∑ ξ kiε

Let f satisfies the assumption I. Let

(

if σ x, x

vi

as

(

 α *rT
LL 
 v 2
 i

(b)

+

)

≤ K 'σ x, x , for i = 1,2,





(

+

,n

)

≤ K "σ x, x .
F

vi = [r1 r2 . . . ri ] .
T

There exist numbers ε , ρ1' and ρ 2 ' such that
+

αi *

i =1 k =1

defined

)

then
(a)

1
p

Lemma 8

vi

(

K > 0 and ε > 0 such that if σ x, x + < ε

i =1 k =1

The

47

) < ε , then ρ

1

≤

r
vi

Proof

(

Since α * = T * y − LT * s = T * y − ∇ 2 f ( x*)s

≤ ρ2 .

)

, if Ti * is the i-th row of T * , then

α i * = Ti * (y − ∇ 2 f ( x*)s ), giving

Proof
By lemma 6, and lemma 7

ρ1' = m ρ1 ≤

ms
vi

T

≤

y s
vi
≤

≤

α i * ≤ Ti * y − ∇ 2 f ( x*)s
M s
+

≤ Ti * Kσ ( x, x ) s

vi

M ρ2 = ρ2

'

By lemma 7 there exist ρ > 0 and ε > 0

Now, in order to proceed with the
main objective, the following abbreviation is
introduced.

r3 . . . ri ]

(

)

such that if σ x, x + < ε and

αi *
s
+
+
≤ K Ti * F σ x, x
≤ Kρ Ti * F ⋅ σ x, x
vi
vi

(

)

(

(a)

v i = [r1 r2

(b)

C = L − L*, C + = L+ − L*,

this proves (a) with K ' = Kρ T * F . Now, to

(c)

α = T y − LT s, α * = T * y − LT * s,

prove (b), let

(d)

β = Cr ,

α * r T
A* = LL  i 2
 v i

consequently,
(e)

  β rT 
 α *rT


− LL 
C = C −  LL
2
 v 2
  v i 
 i

+

Lemma 9










then if Ai * denotes the i -th row of A *

)

48

KomuniTi, Vol.IV No.1 Januari 2012

 α i * rj
Ai * = ∑ 
 v 2
j =1
 i
i

2

Ai * =
2

Ai * =
2

Ai *

2
F

(α i *)2
vi

4

vi






2

vi

2

 α i * rj
= ∑
 v 2
i =1
 i
n






2

∇i

2

∇i

2

2

 β i rj  
β
2
c
r
ij
i
j
2
= ∑ cij −
+ 2  
2
 v  

vi
j =1
 i  

β i2 i 2
2βi i
2
= ∑ cij −
cr +
r
2 ∑ ij j
4 ∑ j
v i j =1
vi j =1

2

( ( ))
n K 'σ (x, x )
+

2

i =1

≤

∇i

= ∑ cij2 −

2 β i2

= ∑ cij2 −

+

2

vi

∇
Lemma 10

Let f satisfies the assumption I and

+

F

≤

vi

4

vi

2

β i2
2

vi

n

= ∑ ∇i

2
F

2

i =1

0< ∇

2
F

n

i =1 j =1

= C

n

−∑

2
F

i =1

β i2

n

i

= ∑ ∑ cij2 − ∑

II. Then the Cholesky update algorithm

(C )

β i2

+

thus,

this proves (b) with K " = n K ' .

satisfies

2

i

(α i *)2

≤ ∑ K ' 2 σ x, x
F

∇i

2


β i rj 

= ∑ cij −
2

v
j =1 
i

i

2

n

A*

∇i

2

i =1

vi

β i2
vi

2

observe that

(1 − θ ) C F + K 'σ (x, x + ),

 1
where θ = 
 C

β 

< C

2

2
F

thus, setting

θ=

Proof

 α * rT
Let ∇ = C − LL 
 v 2
 i

vi

i =1

2

 ∈ (0,1).
v i 

+

β i2

n

0 0 is chosen such that

2

1

numbers ε (h ) and δ (h)

imply that

) C + α σ (x, x )
≤ [1 + α σ (x, x )] C + α σ (x, x )

C + ≤ α 1σ x, x
C+

and

Lm − L * F < δ (h ) and x m − x * < ε (h ) and

+

2

So the Cholesky update function φ ( x, L )

thus, x k +1 − x * ≤ h x k − x * for each k ≥ m .

satisfies the deterioration property and the
So

convergence follows from theorem 1.

x k +1 − x *
xk − x *

≤ h, and since h ∈ (0,1) was

arbitrary, then lim

Corollary 3

k →∞

x k +1 − x *
xk − x *

= 0.

Assume that the hypotheses of Theorem 2
hold. If some subsequence of

{L

k

− L* F

}

converges to zero, then {x k } converges qsuperlinearly to x * .

NUMERICAL RESULTS

In this section, we compare the
numerical results of the Cholesky update
with the BFGS update . The line search

Proof

routine is inexact or satisfies the Wolfe
conditions. In order to assess the value of

50

KomuniTi, Vol.IV No.1 Januari 2012

this approach, numerical test were carried

.ℎ 0 () = 0.1 1 234 5) = 25 +

out on several unconstrained optimization

[−50 ln () ]# .

problems. The test problems chosen were as

%

follows:
1.

= 100

—

+ 1−

2.

= 100

—

+ 1−

= ∑9:
),

8.

.

3.

−3

= 0.0001 (
exp (20 (

−

)

3 234

9:

= ∑9),

= 10

+
−1

− 1.0001] .

6.

= 1−

+

−

.

7.

= ∑++
),

"# $ %

' − () * ,

!

"&

@,

10.

+

−

−

= B C?

−2×

+

[

2+ 3 ,

)+

=

@

)

−1 , 1≤1 ≤

= ∑9@, ?

@

−1

+

)

;ℎ 0

9

10−62+ 1 2−22.
5.

−

)).

− 10

=

4.

−(

= √10

;ℎ 0

9.

.

,

)

A−

− 1A.

= ∑9), D3 − ∑9@, EFG

11−EFG 1−G13 1+ 1−12.
11.

.

@

+

= ∑9), 1 H∑9@, C @ I − 1' .

Tabel 1.
No
1
1
1
2
2
2
3
3

n
2
10
20
2
2
2
2
2

Startingpoint
(-1.2,1)
(-1.2,1)
(-12,1)
(-1.2,1)
(-3.635,5.621)
(639,-0.221
(0,0)
(9,12)

BFGS
Itr
feval
30
188
30
-428
30
728
31
195
132
803
280
1818
5
29
34
201

f min
4.86E-8
4.86E-8
4.86E-8
7.11E-8
3.205E-10
339E4
0.2008
0.2053

Itr
35
35
35
26
35
44
5
14

Cholesky
feval
221
501
851
159
229
263
29
79

f min
6.65E-9
6.65E-9
6.65E-9
5.29E-6
2.03B-5
6.13E-10
0.2008
0.2053

KomuniTi, Vol.IV No.1 Januari 2012

3
4
4
4
5
5
5
6
6
6
7
7
7
8
8
8
8
8
8
9
9
9
9
9
9
10
10
11
11
11
11
11
11

2
2
2
2
2
2
2
3
3
3
3
3
3
10
20
30
10
20
30
20
30
40
20
30
40
10
20
10
20
30
10
20
30

(30,30)
(1,1)
(10,-0.5)
(100,100)
(0,1)
(0.01,5)
(3,-0.5)
(2,4,2.5)
(0.6,6,3.5)
(0.7,5.5,3)
(250,03,0)
(25,-2,0)
(200,0.1,40)
(1,2,3,…)
(1,2,3,..)
(1,2,3,..)
-(1,2,3,..)
-(1,2,3,..)
-(1,2,3,…)
1 – iln
1 – iln
1-i/n
1 - i/2n
1 - il2n
1 - il2n
(1,2,3,..)
(1,2,3,..)
(1,2,3,..)
(1,2,3,..)
(1,2,3,..)
-(1,2,3,..)
-(1,2,3,..)
-(1,2,3,..)

13
27
248
392
3
2
3
10
8
9
14
1
17
5
17
53
5
17
59
25
91
Filed
11
100
101
90
19
3
97
113
2
90
96

81
241
1280
1928.
16
9
16
63
53
59
99
4
123
70
414
1817
70
412
2022
604
3340
Filed
274
3360
4550
1280
269
55
2089
3327
25
2019
3003

3
24
27
32
3
2
3
10
14
12
13
1
15
5
7
265
5
8188
8
9
14
8
12
9
1357
1549
2
6
3
2
4
3

17
201
214
260
16
9
16
68
91
87
91
4
110
70
168
9047
70
199
6031
195
309
145
191
400
402
17143
21175
25
79
107
25
90
106

0.2723
4.87E40
5.49E-11
1.96E-10
0.9999
1
1
2.23E-4
3.06E-6
1.03E-5
7.92E-4
2.8393
3.20E-4
7.62E-5
2.46E-4
2.27E-4
'133E-4
1.68E-4
3.53E-4
6.62E-9
132E-9
6.51E-9
6.18E-9
6.01E-10
2.27E-10
9.2360
4.7698
2.1429
4.6341
7.1311
2.1429
4.6341
7.1311

symmetric property. Cholesky update does

CONCLUSION
Hart

0.1998
1.35E-6
3.69E-7
3.14E-8
0.9999
1
-1
2.09E-4
235E-6
2.68E-5
7.13E-4
2.8393
3.20E-4
7.60E-5
9.47E-4
3.04E-4
1.34E-4
1.67E-4
3.13E-4
2.20E-9
4.82E-10
Filed
3.50E-11
1.63E-9
2.08E-11
29.0596
36.2260
2.1429
4.6341
7.1311
2.1429
4.6341
7.1311

51

update is satisfying the Q-N

not compute the Hessian inverse matrix but

condition but it's not symmetric, and since

this update compute the inverse of Cholesky

the Hessian matrix is symmetric so to

factor with 23 ( total operations to compute

preserve the symmetric property is very

Hessian

important. From equation 11 and equation

operations were n represent the dimension of

12, clear that the cholesky update

the objective function. From table 1, we can

is

satisfying Q-N condition and preserve the

inverse

matrix

is 43 − 23 )

52

KomuniTi, Vol.IV No.1 Januari 2012

see that the Cholesky update with large

dimension

has

a

good

results

REFERENCES
Byrd R. H. and Nocedal J. 1989. A tool for the analysis of quasi-Newton methods with
application to unconstrained minimization. SIAM J. Number. Anal., 26: 727-739
Bartle R. G. 1975. The elements of real analysis. Wiley international edition. USA. 286-330
David G. Luenberger and Yinyu Ye. 2009. linear and Nonlinear Programming. Third Edition.
Springer
Dennis J. E. and Schnabel R. B. 1983. Numerical Methods for Unconstrained Optimization and
Nonlinear Equations. Prentice-Hall, Inc., Englewood Cliffs, New Jersey.
Hart, W. E. 1990. Quasi-Newton methods for sparse Nonlinear system. Memorandum CSM-151,
Department of Computer Science, University of Essex.
Royden, H. L. 1968. Real Analysis. second edition, Macmillan Publishing co. INC. New York.
Saad S.. 2003. Unconstrained Optimization Methods Based On Direct Updating of Hessian
Factors. Ph. D. Dissertation. Gadjah Mada University. Yogyakarta, Indonesia.
Steven C. Chapra. 2005."Applied Numerical Methods With MATLAB for Engineers and
Scientists. McGRAW- HILL INTERNATIONAL EDITION.