2 THE MATHEMATICS OF OPTIMIZATIONBBM

  

The Mathematics of Optimization

• Many economic theories begin with the assumption that an economic agent is seeking to find the optimal value of some function
• – consumers seek to maximize utility
• – firms seek to maximize profit
• This chapter introduces the mathematics common to these problems

  

Maximization of a Function of

One Variable

• Simple example: Manager of a firm wishes to maximize profits

  ) (q f    = f(q)

   Quantity

  * q*

  Maximum profits of * occur at q*

  Maximization of a Function of

One Variable

• The manager will likely try to vary q to see where the maximum profit occurs

  

to q

• – an increase from q

  1 2 leads to a rise in  

  *   

   ₂  = f(q) q

    ₁ Quantity q q q* _{1 2}

  Maximization of a Function of

One Variable

• If output is increased beyond q*, profit will decline
• – an increase from q* to q

  3 leads to a drop in  

  *   

   = f(q) q

    ₃ Quantity q* q ₃

  

Derivatives

• The derivative of  = f(q) is the limit of

  /q for very small changes in q

  h q f h q f dq df dq d h

  ) ( ) ( lim

  1

  1    

   

• The value of this ratio depends on the value of q

  1

  

Value of a Derivative at a Point

• The evaluation of the derivative at the point q = q

  1

  can be denoted

  1 q q dq d

   

• In our previous example,

  1  

  q q dq d

  3  

  q q dq d

     * q q dq d

  First Order Condition for a

Maximum

• For a function of one variable to attain its maximum value at some point, the derivative at that point must be zero

  df  dq q q

  

 *

Second Order Conditions

• The first order condition (d/dq) is a necessary condition for a maximum, but it is not a sufficient condition

   If the profit function was u-shaped, the first order condition would result in q* being chosen and  would be minimized

  * Quantity q*

  

Second Order Conditions

• This must mean that, in order for q* to be the optimum,
• q q dq d
• q q dq d

     for

  and

     for

• Therefore, at q*, d/dq must be decreasing

  

Second Derivatives

• The derivative of a derivative is called a second derivative
• The second derivative can be denoted by

  ) ( " or or

  

2

  2

  2

  2 q f dq f d dq d 

  

Second Order Condition

• The second order condition to represent a (local) maximum is

  ) ( "

  2

  2   

    q q q q

q f

dq d

  

Rules for Finding Derivatives

If then constant, a is 1.

   dx db b

  ) ( ' )] ( [ If then constant, a is 2. x bf dx x bf d b

  

  1 If then constant, is 3. 

   b b bx dx dx b x d 1 ln 4.

  

  5. constant any for ln 

  

Rules for Finding Derivatives

a a a

dx da x x

  x /dx = e x

• – a special case of this rule is de

  

Rules for Finding Derivatives

• Suppose that f(x) and g(x) are two functions of x and f’(x) and g’(x) exist
• Then

  d [ f ( x ) g ( x )]  6. f ' ( x ) g ' ( x )

    dx d [ f ( x ) g ( x )]

   7. f ( x ) g ' ( x ) f ' ( x ) g ( x )   dx

  

Rules for Finding Derivatives

) ( that provided

  )] ( [ ) ( ' ) ( ) ( ) ( ' ) ( ) (

  8.

  2  

       

     x g x g x g x f x g x f dx x g x f d

  

Rules for Finding Derivatives

• If y = f(x) and x = g(z) and if both f’(x) and g’(x) exist, then:

  dy dy dx df dg 9.

      dz dx dz dx dz

• This is called the chain rule. The chain rule allows us to study how one variable (z) affects another variable (y) through its influence on some intermediate variable (x)

  

Rules for Finding Derivatives

• • Some examples of the chain rule include

  ax ax de de d ( ax ) ax ax 10. e a ae

       dx d ( ax ) dx d [ln( ax )] d [ln( ax )] d ( ax )

  11. ln( ax ) a a ln( ax )      dx d ( ax ) dx

  2

  

2

  2 d [ln( x )] d [ln( x )] d ( x )

  1

  2 12. 2 x     

  2

  2 dx d ( x ) dx x x

  

Example of Profit Maximization

• Suppose that the relationship between profit and output is

  2

 = 1,000q - 5q

• The first order condition for a maximum is

  d/dq = 1,000 - 10q = 0 q* = 100

• Since the second derivative is always
• 10, q = 100 is a global maximum

  

Functions of Several Variables

• Most goals of economic agents depend on several variables
• – trade-offs must be made
• The dependence of one variable (y) on a series of other variables (x ,x ,…,x ) is

  1 2 n

  denoted by

  y f ( x , x ,..., x ) 

  1 2 n

Partial Derivatives

• The partial derivative of y with respect to

  x is denoted by

  1 y f

   

or or f or f

x

  1

  1 x x

   

  1

  1

• It is understood that in calculating the partial derivative, all of the other x’s are held constant

Partial Derivatives

• A more formal definition of the partial derivative is

  2 n 2 n f f ( x h , x ,..., x ) f ( x , x ,..., x )

    

  1

  1 lim  h

   x h

  

  1 x ,..., x

  2 n

Calculating Partial Derivatives

  2

  1

  2

  1

  2

  1

  2

  1

  , then ) , ( If 1.

  1

  2

  1 bx ax bx ax bx ax be f x f ae f x f e x x f y

       

      

   

and

If then 2.

  , ) , (

  2

  2

  1

  2

  2

  2

  2

  1

  1

  1

  2

  2

         

  1

  2

  1

  2

  1

  2

  2 cx bx f x f bx ax f x f cx x bx ax x x f y

        and

  

Calculating Partial Derivatives

( , ) ln ln ,

  3. If y f x x a x b x then   

  1

  2

  1

  2 f a f b

    f and f

     

  1

  2 x x x x

  



  1

  1

  2

  2

  

Partial Derivatives

• Partial derivatives are the mathematical expression of the ceteris paribus assumption

• – show how changes in one variable affect

  some outcome when other influences are held constant

  

Partial Derivatives

• We must be concerned with how variables are measured
• – if q represents the quantity of gasoline demanded (measured in billions of gallons) and p represents the price in dollars per gallon, then q/p will measure the change in demand (in billiions of gallons per year) for a dollar per gallon change in price

  

Elasticity

• Elasticities measure the proportional effect of a change in one variable on another
• – unit free
• The elasticity of y with respect to x is

  y x x y y x x y x x y y e x y

      

     

    ,

  

Elasticity and Functional Form

• Suppose that

  y = a + bx + other terms

• In this case,

             

   bx a x b y x b y x x y e x y,

• e

  y,x

  is not constant

• – it is important to note the point at which the elasticity is to be computed

  

Elasticity and Functional Form

• Suppose that

  y = ax b

• In this case,

  b ax x

abx

y x x y e b b x y

        

  1 ,

  

Elasticity and Functional Form

• Suppose that

  ln y = ln a + b ln x

• In this case,

  x y b y x x y e x y ln ln

  ,  

  

  

  

• Elasticities can be calculated through logarithmic differentiation

  

Second-Order Partial Derivatives

• The partial derivative of a partial derivative is called a second-order partial derivative

  2 ( f / x ) f     i f

    ij x x x

    j j i

  

Young’s Theorem

• Under general conditions, the order in which partial differentiation is conducted to evaluate second-order partial derivatives does not matter

  f f  ij ji

  

Use of Second-Order Partials

• Second-order partials play an important role in many economic theories

• • One of the most important is a variable’s

  own second-order partial, f

  ii

• – shows how the marginal influence of x on

  i ) changes as the value of x i i y(y/x increases < 0 indicates diminishing

• – a value of f

  ii marginal effectiveness

  

Total Differential

• Suppose that y = f(x

  ,x ,…,x )

  1 2 n

• If all x’s are varied by a small amount, the total effect on y will be

  f f f    dy dx dx dx ...

     

  1

  2 n x x x

    

  1 2 n dy f dx f dx ... f dx

     

  1

  1

  2 2 n n

First-Order Condition for a Maximum (or Minimum)

• A necessary condition for a maximum (or minimum) of the function f(x ,x ,…,x ) is

  1 2 n

  that dy = 0 for any combination of small changes in the x’s

• The only way for this to be true is if

  f f ... f    

  1 2 n

• A point where this condition holds is called a critical point
• Suppose that y is a function of x

• 1)
• (x
• 2)
• 10
• 2x
• x
• 4x
• 5

• First-order conditions imply that

  1  

  2

  1

  2

  OR

         x x y x x y

  1     

  1

  2

  2

  2

  2

  2

  4

  2

  22

  1

  12

  y = - x

  2

  2

  2

  1

  2 y = - (x

  and x

  1

  

Finding a Maximum

  x x

  

Production Possibility Frontier

  2

  2

• Earlier example:

  2x + y = 225

  2

  2

• Can be rewritten: f(x,y) = 2x + y - 225 = 0
• Because f = 4x and f = 2y, the opportunity

  x y

  cost trade-off between x and y is

  dy f x x

  4

  2

  

x

     dx f 2 y y y

  

Implicit Function Theorem

• It may not always be possible to solve implicit functions of the form g(x,y)=0 for unique explicit functions of the form y = f(x)
• – mathematicians have derived the necessary conditions
• – in many economic applications, these conditions are the same as the second-order conditions for a maximum (or minimum)

The Envelope Theorem

• The envelope theorem concerns how the optimal value for a particular function changes when a parameter of the function changes
• This is easiest to see by using an example

  

The Envelope Theorem

• Suppose that y is a function of x

  y = -x

  

2

• ax
• For different values of a, this function represents a family of inverted parabolas
• If a is assigned a specific value, then y becomes a function of x only and the value of x that maximizes y can be calculated

  

The Envelope Theorem

Optimal Values of x and y for alternative values of a

  The Envelope Theorem As a increases, the maximal value for y (y*) increases The relationship between a and y is quadratic

The Envelope Theorem

• Suppose we are interested in how y* changes as a changes
• There are two ways we can do this
• – calculate the slope of y directly
• – hold x constant at its optimal value and calculate y/a directly

The Envelope Theorem

• To calculate the slope of the function, we must solve for the optimal value of x for any value of a

  

dy/dx = -2x + a = 0

x* = a/2

• Substituting, we get

  y* = -(x*)

  2

  2

• + a(x*) = -(a/2)

• a(a/2)

  y* = -a

  2 /4 + a

  2 /2 = a

  2 /4

The Envelope Theorem

• Therefore,

  dy*/da = 2a/4 = a/2 = x*

• But, we can save time by using the envelope theorem
• – for small changes in a, dy*/da can be computed by holding x at x* and calculating y/ a directly from y

The Envelope Theorem

  y/ a = x

• Holding x = x*

  y/ a = x* = a/2

• This is the same result found earlier

  

The Envelope Theorem

• The envelope theorem states that the change in the optimal value of a function with respect to a parameter of that function can be found by partially differentiating the objective function while holding x (or several x’s) at its optimal value
• dy y

   { )} * x x ( a   da a

  

  

The Envelope Theorem

• The envelope theorem can be extended to the case where y is a function of several variables

  

y = f(x ,…x ,a)

1 n

• Finding an optimal value for y would consist of solving n first-order equations

  = 0 (i = 1,…,n) i

  y/x

The Envelope Theorem

• Optimal values for theses x’s would be determined that are a function of a

  x * = x *(a)

  1

  1 x * = x *(a)

  2

  2

.

  

.

.

x *= x *(a)

n n

The Envelope Theorem

• Substituting into the original objective function yields an expression for the optimal value of y (y*)
• (a), x
• (a),…,x
• (a),a]
• Differentiating yields
• 2

  y* = f [x

  1

  2

  n

  a f da dx x f

da

dx

x f da dx x f da dy n n

      

       

      

  ...

  2

  1

  1

  )} ( * {

  

The Envelope Theorem

• Because of first-order conditions, all terms except f/a are equal to zero if the x’s are at their optimal values
• Therefore,
• a x x a f da dy

     

Constrained Maximization

• What if all values for the x’s are not feasible?

• – the values of x may all have to be positive

• – a consumer’s choices are limited by the amount of purchasing power available
• One method used to solve constrained maximization problems is the Lagrangian multiplier method
• Suppose that we wish to find the values of x

  

Lagrangian Multiplier Method

  1

  , x

  2

  ,…, x

  n

  that maximize

  y = f(x

  1 , x

  2 ,…, x n

  )

  subject to a constraint that permits only certain values of the x’s to be used

  g(x

  1 , x

  2 ,…, x n

  ) = 0

  

Lagrangian Multiplier Method

• The Lagrangian multiplier method starts with setting up the expression

  L = f(x , x ,…, x , x ,…, x )

  1 2 n

  1 2 n ) + g(x

  where  is an additional variable called a Lagrangian multiplier

• When the constraint holds, L = f because g(x , x ,…, x ) = 0

  1 2 n

• First-Order Conditions

  L/x n

  2 ,…, x n

  1

, x

  L/ = g(x

  .

  n = 0 .

  = f

n

  2 = 0 .

  2 + g

  2 = f

  1 = 0 L/x

  1 + g

  1 = f

  L/x

  

Lagrangian Multiplier Method

• g

  ) = 0

  

Lagrangian Multiplier Method

• The first-order conditions can generally be solved for x

  1

  , x

  2

  ,…, x

  n and 

• The solution will have two properties:
• – the x’s will obey the constraint
• – these x’s will make the value of L (and therefore f) as large as possible

  

Lagrangian Multiplier Method

• The Lagrangian multiplier () has an important economic interpretation
• The first-order conditions imply that

  f /-g = f /-g =…= f /-g

  1

  1

  2 2 n n = 

• – the numerators above measure the marginal benefit that one more unit of x will have for the

  i function f

• – the denominators reflect the added burden on the constraint of using more x

  i

  

Lagrangian Multiplier Method

• At the optimal choices for the x’s, the ratio of the marginal benefit of increasing

  x to the marginal cost of increasing x i i should be the same for every x

  is the common cost-benefit ratio for all

of the x’s marginal benefit of x i

    marginal cost of x i

  

Lagrangian Multiplier Method

• If the constraint was relaxed slightly, it would not matter which x is changed
• The Lagrangian multiplier provides a measure of how the relaxation in the constraint will affect the value of y

    provides a “shadow price” to the constraint

  

Lagrangian Multiplier Method

• A high value of  indicates that y could be increased substantially by relaxing the constraint
• – each x has a high cost-benefit ratio
• A low value of  indicates that there is not much to be gained by relaxing the constraint

   =0 implies that the constraint is not binding

  

Duality

• Any constrained maximization problem has associated with it a dual problem in constrained minimization that focuses attention on the constraints in the original problem

Duality

• Individuals maximize utility subject to a budget constraint
• – dual problem: individuals minimize the expenditure needed to achieve a given level of utility
• Firms minimize the cost of inputs to produce a given level of output

• – dual problem: firms maximize output for a

  given cost of inputs purchased

Constrained Maximization

• Suppose a farmer had a certain length of fence (P) and wished to enclose the largest possible rectangular shape
• Let x be the length of one side
• Let y be the length of the other side

• • Problem: choose x and y so as to maximize

  the area (A = x·y) subject to the constraint

  that the perimeter is fixed at P = 2x + 2y

Constrained Maximization

• Setting up the Lagrangian multiplier

  L = x·y + (P - 2x - 2y)

• The first-order conditions for a maximum are

  L/x = y - 2 = 0 L/y = x - 2 = 0 L/ = P - 2x - 2y = 0

  

x = y = P/4

 = P/8

  

Constrained Maximization

• Since y/2 = x/2 = , x must be equal to y
• – the field should be square
• – x and y should be chosen so that the ratio of marginal benefits to marginal costs should be the same
• Since x = y and y = 2, we can use the constraint to show that

  

Constrained Maximization

• Interpretation of the Lagrangian multiplier

• – if the farmer was interested in knowing how

  much more field could be fenced by adding an extra yard of fence,  suggests that he could find out by dividing the present perimeter (P) by 8
• – thus, the Lagrangian multiplier provides

  information about the implicit value of the

  constraint

  

Constrained Maximization

• Dual problem: choose x and y to minimize the amount of fence required to surround the field

  

minimize P = 2x + 2y subject to A = x·y

• Setting up the Lagrangian:

  D = 2x + 2y + 

  D (A - xy)

L

Constrained Maximization

• First-order conditions:

  D D ·y = 0 L

  /x = 2 -  D D

  ·x = 0 L /y = 2 - 

  D D = A - x·y = 0

  L /

• Solving, we get

  1/2

x = y = A

D -1/2

  ) = 2A

• The Lagrangian multiplier (

  Envelope Theorem &

Constrained Maximization

• Suppose that we want to maximize

  y = f(x ,…,x ;a) 1 n

  subject to the constraint

  

g(x ,…,x ;a) = 0

1 n

• One way to solve would be to set up the

  Lagrangian expression and solve the first- order conditions

  Envelope Theorem &

Constrained Maximization

• Alternatively, it can be shown that
• ,…,x *;a)

  1 n dy*/da = L/a(x

• The change in the maximal value of y that results when a changes can be found by partially differentiating L and evaluating the partial derivative at the optimal point

  

Inequality Constraints

• In some economic problems the constraints need not hold exactly
• For example, suppose we seek to maximize y = f(x

  1

  ,x

  

2

  ) subject to

  g(x

  1 ,x

  

2

)  0, x

  1  0, and x

  2  0

  

Inequality Constraints

• One way to solve this problem is to introduce three new variables (a, b, and

  c) that convert the inequalities into

  equalities

• To ensure that the inequalities continue to hold, we will square these new variables to ensure that their values are positive

  

Inequality Constraints

  2

g(x ,x ) - a = 0;

  1

  2

  2

x - b = 0; and

  1

  2

x - c = 0

  2

• Any solution that obeys these three equality constraints will also obey the inequality constraints
• We can set up the Lagrangian

  

Inequality Constraints

  L = f(x

• b

  1 ,x

  2 ) + 

  1 [g(x

  1 ,x

  2

) - a

  2 ] + 

  2 [x

  1

  2 ] + 

  3 [x

  2

  2 ]

• c
• This will lead to eight first-order conditions

Inequality Constraints

  

= f g = 0

  1

  1

  1

  1

  2 L/x +  + 

= f g = 0

  2

  1

  1

  2

  3 L/x +  +  = 0

  1 L/a = -2a = 0 L/b = -2b

  2 = 0

  3 L/c = -2c

  2 = g(x ,x ) - a = 0

  1

  1

  

2

L/

  2 = x - b = 0

  2

  1 L/

  2 = x - c = 0 L/

  3

  2

  

Inequality Constraints

• According to the third condition, either a or 

  1

  = 0

• – if a = 0, the constraint g(x

  1 ,x

  2 ) holds exactly

  1 = 0, the availability of some slackness of the constraint implies that its value to the objective function is 0

• – if 
• Similar complemetary slackness relationships also hold for x

  1

  and x

  2

  

Inequality Constraints

• These results are sometimes called

Kuhn-Tucker conditions

• – they show that solutions to optimization

  problems involving inequality constraints will differ from similar problems involving equality constraints in rather simple ways

• – we cannot go wrong by working primarily

  with constraints involving equalities

  Second Order Conditions -

Functions of One Variable

• Let y = f(x)
• A necessary condition for a maximum is that

  dy/dx = f ’(x) = 0

• To ensure that the point is a maximum, y must be decreasing for movements away from it

  

Second Order Conditions -

Functions of One Variable

• The total differential measures the change in y

  

dy = f ’(x) dx

• To be at a maximum, dy must be decreasing for small increases in x
• To see the changes in dy, we must use the second derivative of y

  

Second Order Conditions -

Functions of One Variable d [ f ' ( x ) dx ]

  2

  2 d y dx f " ( x ) dx dx f " ( x ) dx

       dx

  2

  2

• Note that d

  y < 0 implies that f ’’(x)dx < 0

  2

• Since dx must be positive, f ’’(x) < 0
• This means that the function f must have a concave shape at the critical point

Second Order Conditions - Functions of Two Variables

• Suppose that y = f(x

  1 , x

  2 )

• First order conditions for a maximum are

  y/x

  1 = f

  1 = 0 y/x

  2 = f

  2 = 0

• • To ensure that the point is a maximum, y

  must diminish for movements in any direction away from the critical point

Second Order Conditions - Functions of Two Variables

• The slope in the x direction (f ) must be

  1

  1 diminishing at the critical point

• The slope in the x direction (f ) must be

  2

  2 diminishing at the critical point

• • But, conditions must also be placed on the

  cross-partial derivative (f = f ) to ensure

  12

  21 that dy is decreasing for all movements through the critical point

Second Order Conditions - Functions of Two Variables

• The total differential of y is given by

  dy = f dx + f dx

  1

  1

  2

  2

• The differential of that function is

  2 d y = (f dx + f dx )dx + (f dx + f dx )dx

  11

  1

  12

  2

  1

  21

  1

  22

  2

  2

  2 d y = f dx + f dx dx + f dx dx + f dx

  11

  12

  12

  

2

  1

  21

  1

  2

  22

  22

• By Young’s theorem, f = f and

  12

  21

  2 d y = f dx + 2f dx dx + f dx

  11

  12

  12

  1

  2

  22

  22

  

Second Order Conditions -

Functions of Two Variables

  2

d y = f dx + 2f dx dx + f dx

  11

  12

  12

  1

  2

  22

  22

• For this equation to be unambiguously

  negative for any change in the x’s, f and f

  11

  22 must be negative

  2

• If dx = 0, then d y = f dx

  2

  11

  12

  2 y < 0, f < 0

• – for d

  11

  2

• If dx = 0, then d y = f dx

  1

  22

  22

  2

• – for d y < 0, f < 0

  22

  

Second Order Conditions -

Functions of Two Variables

  2 d y = f dx + 2f dx dx + f dx

  11

  12

  12

  1

  2

  22

  22

  2 nor dx is zero, then d y will be

  1

  2

• If neither dx unambiguously negative only if

  

f f - f > 0

  11

22 122

and f ) must

  11

  22

• – the second partial derivatives (f be sufficiently negative so that they outweigh

  any possible perverse effects from the cross-

  partial derivatives (f = f )

  12

  21

  

Constrained Maximization

• Suppose we want to choose x and x to

  1

  2

  maximize

  

y = f(x , x )

  1

  2

• subject to the linear constraint

  

c - b x - b x = 0

  1

  1

  2

  2

• We can set up the Lagrangian

L = f(x , x x - b x )

  1

  2

  1

  1

  2

  2

) + (c - b

• The first-order conditions are
• b
• b

• - b

  2 = 0

  22 dx

  2

  1 dx

  12 dx

  12

  11 dx

  2 y = f

  d

  2 x

  1

  1 x

  2 = 0 c - b

  2

  1 = 0 f

  1

  f

  

Constrained Maximization

• To ensure we have a maximum, we must use the “second” total differential
• 2f
• f

  22

  

Constrained Maximization

• Only the values of x and x that satisfy

  

1

  2 the constraint can be considered valid alternatives to the critical point

• Thus, we must calculate the total differential of the constraint
• b dx - b dx = 0

  1

  1

  2

  2

dx = -(b /b )dx

  2

  1

  2

  1

• • These are the allowable relative changes

  in x and x

  1

  2

Constrained Maximization

• Because the first-order conditions imply that f
• Since
• 2f
• f

  1 /f

  22 we can substitute for dx

  12

  22 )dx

  12 /f