M00316

DIFFICULTY IN OPTIMIZATION FUNCTIONS
OF MATLAB AND HOW TO ANALYZE
H.A Parhusip
Center of Applied Science and Mathematics
Universitas Kristen Satya Wacana
Jl. Dipenegoro 52-60, Salatiga, 50711, Central Java
Telephone number : 0062-298-321212
Email address : hannaariniparhusip@ yahoo.co.id
Abstract
This paper presents an example of using optimization function in MATLAB.
The used data is considered as a smooth function of 3 variables. Unlike in a literature,
one needs to design a smooth function in order to start with an optimization. After the
parameters are obtained the optimization is employed to obtain the minimizer. The
obtained objective function is noncoercive. Therefore the minimizer is obtained by
solving the nonlinear system of the Lagrangian function which is constructed as a
perturbed objective function. The optimizer can not be considered as the best solution
since the Hessian is not positive definite.

Keywords : noncoercive function, Jacobian, least square, Hessian, singularity

Introduction

Understanding output of MATLAB’s program is not an easy
task. After using any functions in MATLAB correctly, one needs to
relate with an expected result. Therefore theoretical background that may
be used in the functions must be known.
MATLAB is also used for undergraduate students for many
subjects in particular computation mathematics, numerical methods, data
analysis, linear algebra, nonlinear optimization and mathematical
modeling which is based on differential equations. Having very much
time in theories in those subjects, students are very lack on applications.
On the other hand, giving practical applications by too much in
programming, students have very low theoretical background for those
1

subjects and students could not also describe formally mathematical
reasons in the given output by MATLAB.
There are many authors addressed on developing the MATLAB
code for a particular purposes. For data single directional (azimuthal) in
geosciences for instance, Jones [1] proposed a MATLAB script since for
this special purpose one may not have the related function from the
standard MATLAB Toolbox. The lsqnonlin (one of optimization

functions in MATLAB) can not be used for the directional azimuthal
data. The given data is shown in Table 1. The goal of this measurement is
to identify the maximum protein that can be obtained from various
observations.
Table 1
The used data for the optimization study (Observed by. Y.Martono at Lab. Chemistry,
SWCU,2011)
(B)

Percentage of
yeast
(Y)

Protein
Day
1

Protein
Day
2

Protein
Day
3

Protein
Day
4

1

5

0.714

0.75

1.76

3.22

1

7.5

1.33

1.88

0.21

1.33

1

12.5

0.88

1.38

0.93

2.38

Reseach Method
Some authors refer to linear regression to model percentage
change over time (Suissa,et.all,1989). In the case of molecular protein,
this approachs can not be applied since one needs modeling threedimensional detail by homology (Sander and Scheider,1991). Since the
number of the given data is small, The first approach in this paper is to
present the percentage protein as a smooth function and then we
introduce the optimization procedures. Let us introduce dimensionless
variables,i.e t : time ; B : percentage of beans ;Y: percentage of yeast, P :
percentage of protein. We assume that the given data will be a continuous
function as

 Y Y
P  t,     t 
 B  B


,  ,  unknown parameters

(1)

Note that we have made data into dimensionless to proceed futher
computations. Using the standard least square means that we have to


  Y    

minimize the residual function
R   Pi ,da ta    i  t i   .

  Bi 

i 1




*
Following the standard optimization to find the critical x  [ ,  ]T
  
means that we have to solve the system obtained by R : g ( x)  0 . In
2

n

more details, this leads to solve
Y
  Pi ,da ta  i
i 1
 Bi
n

Y
  Pi ,da ta  i
i 1
 Bi
n

   Yi
 t i ln 

 Bi


n

Y
    i
 i 1  Bi

n
Y
 
 t i ln t i    i
i 1  Bi




R
R
 0 and
 0 simultaneously,i.e






2

Y
t i2  ln  i
 Bi


 : g 1 ( ,  )  0 ;



 2
 t i ln t i  0 := g 2 ( ,  ) =0.

2

(2a)
(2b)

Any algorithm with an iterative procedure requires an initial
guesses of solutions. There are several well-known algorithms such as
Newton-method, Broyden method, trust-region and using evolutionary
algorithm [2]. The last algorithm transforms the system of equations into
a multi-objective optimization problem. The ill-posed nonlinear
equations may also occur. Buong and Dung [3] have provided a
theoretical study on this particular problem by doing a regularization on
the problem which based on Tikhonov regularization method. However
the problem was designed in the variational form which was too much
beyond from our study in this paper. Thus, this paper will use the

3

standard Newton method. The Newton method solves a general system
  
g ( x)  0 with a formula





1  


 
x ( k 1)  x ( k )  g ( x ( k ) )  g ( x ( k ) )

(3)
  (k)
which provides a nonsingularity of matrix Jacobian g ( x ) on each
iteration step. The formula (3) is employed to Eq. (2a)-(2b) by deriving
their derivatives manually each term on these equations. Therefore one
needs
 g1
  (k)
g ( x )   g2
 

with

g1


g1

g 2


Y
=   Pi ,da ta  i
i 1
 Bi
n

g1


n
Y
=   Pi ,da ta  i
B
i 1
 i



 x( ( k ) , ( k ) )

(4)

    Yi
 t i  ln 

   Bi


2
2
n

   2  Yi  t i2   ln  Yi   ,

 
  B 
i 1  Bi 

  i 

2

n
Y
   Yi 
 t i ln   ln t i  2  i
i 1  Bi
 Bi 




 2   Yi 
 t i ln   ln t i
 Bi 

2

, g2   Pi ,da ta  Yi  t i ln t i ln Yi  2  Yi  t i2  ln t i ln Yi ,
n

i 1

g 2






 Bi 

Y
=   Pi ,da ta  i
i 1
 Bi
n



n

Bi

i 1



2

 Bi 

n
 
Y
 t i ln t i 2  2  i
i 1  Bi






Bi

 2
 t i ln t i 2 .

2

If we consider carefully, the Jacobian matrix (4) is also the
Hessian matrix of R. Therefore the observation of singularity of (4) leads
to the observation of positive definiteness of the Hessian matrix R. Thus
it inspires us to use the condition that Hessian matrix must be positive
definite in order to work with Eq.(3). In a short expression means that we
have to satisfy

g1


 0 and

g1 g 2
 

g1 g 2


- 

> 0.

Observe

that

the


second condition is the determinant of g which is not allowed to be

zero since it guarantees the existence of the invers of g on each

interation, i.e

g ( x )
(k)

1



g1 g 2
 

1
 g1

 g2
 g 2
 

g 2


 g1 
g1 
  x( ( k ) ,  ( k ) )

(5)

Therefore we only use the Eq.(5) to have the existence of Eq.(4). Thus in
order to use Newton method, we need to include

g1 g 2
 

g1 g 2


- 

>0

for each iteration. Since we seek the solution of R : g ( x)  0 , then

 



( k 1)
)  0.
the iteration stops as g ( x

 

Result and Discussion

Using the lsqnonlin,m from MATLAB, the function (1) is obtained as
Y
 Y Y
P  t,     t    
 B
 B  B


0.4325

t 1.2777 ,

(8)

with the error 0.9008 %. This result is considerable good. Since the data
are not too large, we may make a list of each value of the approximation
compared to the observation (shown in Table 1) and it is depicted in
Figure 1. Now,one may proceed the optimization goal ,i.e max P subject
to the constraints

Y 
g1 (Y, B, t )     0
 B

and g 2 (Y, B, t ) 

t > 0.

Unfortunately, the standard procedure in literature for an optimization is
a minimization case as shown by Peressini,et.all [4] and the function in
5

MATLAB is usually proposed a minimization solver. Therefore one may
set up the optimization model as :

(P )

0.4325


Y
t 1.2777 subject to
 min f ( x)   

B


Y
~ (Y, B, t )  
~ (Y, B, t )  t  0.
g
   0, g
1
2

 B


MATLAB provides fmincon.m function to solve this minimization problem.
Up to this step, the program can not give a reasonable solution. One reason is
due to the property of the objective function which is not coercive
(i.e lim f ( x)   ) If the objective function is coercive then it is guaranteed at
x 



least there exists one minimizer (Peressini,et.all,1988). How do we prove ?.
Table 1 Comparisan measurement
(first
column) and approximation
(second column) of protein
(dimensionless)
0.2217
0.4130
0.2733
0.2329
0.5839
0.4286
0.5466
0.0652
0.2888
1.0000
0.4130
0.7391

0.1145
0.1364
0.1701
0.2775
0.3307
0.4125
0.4658
0.5551
0.6924
0.6728
0.8018
1.0000

Figure 1. The illustration of
Y 
 Y  Y 
P  t,     t    
B
B
 B
  



Y
Note that lim f ( x)  lim   
x 
Y , B , t 
 B



0.4325

t 1.2777

0.4325

t 1.2777 which tends to minus

infinity. Thus it is not coercive. What can we do ?. We can not change the

objective function since it has been obtained from the previous optimization
process. Thank to Peressini(et all,1988), that it offers the idea to perturb the
objective function. More specifically, for each   0, define



2
f  ( x )  f ( x)   x .

Unfortunately,

we

need

also

property

(7)

that


f ( x)

is

convex

(Peressini,et.all,1988,page 52) i.e


  

 
f ( x)  f ( x)  ( y  x)  f ( y) , y, x in a convex set subset of R n . (8)
Let us try to study this condition for all values in the given domain and we

Y
write into more general form,i.e f ( x) =    t  . As a consecuence, one
 B



needs to compute f ( x )   Y 1 B  t 



YB

( 1)

Y

 1 

t


 Y   1 
  t  .
 B

T

Using the values, in the given domain we obtain that the
 
condition (8) is satisfied for y  x . This is shown on Figure 2. Thus, the


Y
property (8) is satisfied and hence the function f ( x) =    t  is
 B
2
convex. Clearly, the x is a quadratic function which is known to be


convex. Therefore the perturbed function defined Eq.( 7) is convex. We


are done to show that f  ( x) is convex. The proof that f  ( x) is coercive
is not shown here for simplicity, which is mentioned into detail in
Peressini (page 229). The optimization problem becomes
(P  )



 2
Y 
 
 min f ( x)    t   x subject to

B


Y
~ (Y, B, t )  
~ (Y, B, t )  t  0.
g
   0, g
2
 1
 B


7


  

f ( x)  f ( x)  ( y  x)  f ( y) obtained by each value of
 
y > x from the given domain.

Penalty Method with a noncoercive objective function f ( x)
Figure 2.
Illustration of

We need to construct the Lagrangian L ( x,  ) for P  as follows



m



L ( x,  )  f  ( x)   i g~i ( x) , m = the number of constraints.
i 1

This Lagrangian becomes an unconstraint objective function with additional 2
unknown parameters.We can use the function fminunc.m in MATLAB by
defining the Lagrangian function as the objective function. We have tried 5 sets
of initial guesses and the observation shows unreasonable answers. Analytic
observation will explain why the function fminunc.m and fmincon.m do not
give unique solutions.

One needs to construct L ( x,  )  0 analytically and solve it by





solver of nonlinear system equations. We have

 
L ( x,  )
: g 1 ( x,  )   Y 1 B  t   2Y  1 B 1  0 ;
Y
 
L ( x,  )
 
: g 2 ( x,  )  Y B ( 1) t   2B  1 YB  2  0 ;
B
 
 
L ( x,  )
 g 3 ( x,  )  Y B  t  1  2  t  2  0,
t



 
 
L ( x,  )
Y
 g 4 ( x,  )     0 ;
1
 B



(9a)
(9b)
(9c)
(9d)

 
 
L ( x,  )
 g 5 ( x,  )  t  0.
 2

(9e)


 
T
We write the system as g ( x)  g1 g 2 g 3 g 4 g 5   0 . One obtains

t * =0; Y*  0 respectively.

directly from Eq.(9e) and Eq.(9d) that

Substitute these result to Eq,(9c), one yields *2  0 . As a result, 1*  0

to satisfy Eq.(9b). On the other hand B*  0 since it will make Eq. (9b) is

undefined if B*  0 . Therefore B*  0 and it is free variable. Thus the
system

has

infinitely

many

solutions

in

the

form

(Y* , B* , t * , 1* , *2 )  (0, a ,0,0,0) with a  0 . Moreover this conclusion is

practically useless. However this explains us that the Newton method
 
will not converge if this method is used blindly. By forcing g ( x) tends



to zero (as shown in the Figure) , the solution x * is not a real vector after 747

 

iterations though the method works well by showing that g ( x) tends to zero
as depicted by Figure 3. Thus the Newton method fails which agrees with

analytic result.

 

Figure 3. Illustration g ( x) for each iteration

9

Conclusion
This paper has presented an example of using the optimization solver
provided by MATLAB. The used data is considered as a smooth function of 3
variables. After the parameters are obtained the optimization is employed to
obtain the minimizer of the noncoercive objective function.
The minimizer is obtained by solving the nonlinear system of the
Lagrangian function which is constructed as a perturbed objective function. The
optimizer can not be considered as the best solution since the Hessian is not
positive definite. One may avoid the computation of gradient by using ant
colony algorithm, particle swarm algorithm (Rao,2009).
Aknowledgement

The author gratefully acknowledge to Yohanes Martono for supporting
his data to this numerical work with MATLAB .
References
Buong,N and Dung. D, (2009).Regularization for a Common Solution of
a System of Nonlinear Ill-Posed Equations, Int. Journal of Math.
Analysis,3(34), 1693-1699.
Grosan, C and Abraham, A,(2008). Multiple Solutions for a System of
Nonlinear Equations, International Journal of Innovative Computing,
Information and Cotrol, x( x), ISSN 1349-4198.
Jones, T.A, (2006). MATLAB functions to analyze directional
(azimuthal) data-I:Single-sample inference, Computers & Geosciences
32 (2006) 166-175.
Peressini, A.L, Sullivan, F.E, Uhl,J, (1988). The Mathematics of
Nonlinear Programming, Springer Verlag, New York, Inc.
Rao, S.S. (2009). Engineering Optimization, Theory and Practice, John
Wiley & Sons, Inc., Hoboken, New Jersey.
Sander C, and Scheider,R, (1991). Database of homology-derived protein
structures and the structural meaning of sequence alignment, Protein,
9(1),56-68.
Suissa S, Levinton C, Esdaile JM. (1989). Modeling percentage change:
a potential linear mirage,J.Clin. Epidemiol, 42(9),843-848.