An Introduction to Stochastic Optimal Control

  ❆♥ ■♥tr♦❞✉❝t✐♦♥ t♦ ❙t♦❝❤❛st✐❝ ❖♣t✐♠❛❧ ❈♦♥tr♦❧

  ❍❡rr② P✳ ❙✉r②❛✇❛♥ ❉❡♣t✳ ♦❢ ▼❛t❤❡♠❛t✐❝s✱ ❙❛♥❛t❛ ❉❤❛r♠❛ ❯♥✐✈❡rs✐t②✱ ❨♦❣②❛❦❛rt❛

  ❏✉♥❡ ✸✱ ✷✵✶✺

  ❖✉t❧✐♥❡

  ▼♦t✐✈❛t✐♦♥ ❙t♦❝❤❛st✐❝ ❖♣t✐♠❛❧ ❈♦♥tr♦❧ ❙♦♠❡ ❊①❛♠♣❧❡s ❖✉t❧♦♦❦✿ ❇❡②♦♥❞ ❇r♦✇♥✐❛♥ ▼♦t✐♦♥

  ▼♦t✐✈❛t✐♦♥

  ❚r❛❝❦✐♥❣ ❛ ❉✐✛✉s✐♦♥ P❛rt✐❝❧❡ ❯♥❞❡r ❛ ▼✐❝r♦s❝♦♣❡

❙✐t✉❛t✐♦♥ ✿ ✇❡ st✉❞② s♦♠❡ ❦✐♥❞ ♦❢ ♣❛rt✐❝❧❡s ✐♥ ❞❡t❛✐❧ ❜② ③♦♦♠✐♥❣ ✐♥ ♦♥ ♦♥❡ ♦❢

  t❤❡ ♣❛rt✐❝❧❡s✱ ✐✳❡✳ ✇❡ ✐♥❝r❡❛s❡ t❤❡ ♠❛❣♥✐✜❝❛t✐♦♥ ♦❢ t❤❡ ♠✐❝r♦s❝♦♣❡ ✉♥t✐❧ ♦♥❡ ♣❛rt✐❝❧❡ ✜❧❧s ❛ ❧❛r❣❡ ♣❛rt ♦❢ t❤❡ ✜❡❧❞ ♦❢ ✈✐❡✇✳

  

Pr♦❜❧❡♠ ✿ t❤❡ r❛♥❞♦♠ ♠♦✈❡♠❡♥t ♦❢ t❤❡ ♣❛rt✐❝❧❡ ❝❛✉s❡s ✐t t♦ r❛♣✐❞❧② ❧❡❛✈❡ ♦✉r

  ✜❡❧❞ ♦❢ ✈✐❡✇✳ ❙♦✱ ✇❡ ❤❛✈❡ t♦ ❦❡❡♣ ♠♦✈✐♥❣ ❛r♦✉♥❞ t❤❡ ❝♦✈❡r s❧✐❞❡ ✐♥ ♦r❞❡r t♦ tr❛❝❦ t❤❡ ♠♦t✐♦♥ ♦❢ t❤❡ ♣❛rt✐❝❧❡✳

  

❍♦✇ t♦ ❞♦ ✿ ✇❡ ❛tt❛❝❤ ❛♥ ❡❧❡❝tr✐❝ ♠♦t♦r t♦ t❤❡ ♠✐❝r♦s❝♦♣❡ s❧✐❞❡ ✇❤✐❝❤ ❛❧❧♦✇s

  ✉s t♦ ♠♦✈❡ t❤❡ s❧✐❞❡ ❛r♦✉♥❞✳ t

  ▼♦❞❡❧ ✿ ▲❡t z ❜❡ t❤❡ ♣♦s✐t✐♦♥ ♦❢ t❤❡ s❧✐❞❡ r❡❧❛t✐✈❡ t♦ t❤❡ ❢♦❝✉s ♦❢ t❤❡

  ♠✐❝r♦s❝♦♣❡✳ ❚❤❡♥✱ ✇❡ ❝❛♥ ✇r✐t❡ dz t = βu t , t dt

  ✇❤❡r❡ u ✐s t❤❡ ✈♦❧t❛❣❡ ❛♣♣❧✐❡❞ t♦ t❤❡ ♠♦t♦r ❛♥❞ β > ✵ ✐s ❛ ❣❛✐♥ ❝♦♥st❛♥t✳ ❚❤❡ ♣♦s✐t✐♦♥ ♦❢ t❤❡ ♣❛rt✐❝❧❡ r❡❧❛t✐✈❡ t♦ t❤❡ s❧✐❞❡ ✐s ♠♦❞❡❧❧❡❞ ❜② ❛ ❇r♦✇♥✐❛♥ t ♠♦t✐♦♥ x ✳ ❍❡♥❝❡✱ t❤❡ ♣♦s✐t✐♦♥ ♦❢ t❤❡ ♣❛rt✐❝❧❡ r❡❧❛t✐✈❡ t♦ t❤❡ ♠✐❝r♦s❝♦♣❡ t + z t ❢♦❝✉s ✐s x ✳

  

●♦❛❧ ✿ t♦ ❝♦♥tr♦❧ t❤❡ s❧✐❞❡ ♣♦s✐t✐♦♥ t♦ ❦❡❡♣ t❤❡ ♣❛rt✐❝❧❡ ✐♥ ❢♦❝✉s✱ ✐✳❡✳ t♦ ❝❤♦♦s❡

  u + z t ✐♥ ♦r❞❡r t❤❛t x t t st❛②s ❝❧♦s❡ t♦ ③❡r♦✳

  

✿ tr❛❝❦✐♥❣ ❛ ❇r♦✇♥✐❛♥ ♣❛rt✐❝❧❡

  ❋♦r♠❛❧✐③❛t✐♦♥ ♦❢ t❤❡ Pr♦❜❧❡♠ ❈♦st ❋✉♥❝t✐♦♥❛❧ ✿ Z T Z T ! !

  ✶ ✶

  ✷ ✷

  J dt u dt T [u] = pE (x t + z t ) + qE , t T T

  ✵ ✵

  ✇❤❡r❡ p ❛♥❞ q ❛r❡ ♣♦s✐t✐✈❡ ❝♦♥st❛♥ts✳

  ❋✐rst t❡r♠ ✿ t✐♠❡✲❛✈❡r❛❣❡ ♦❢ ♠❡❛♥ sq✉❛r❡ ❞✐st❛♥❝❡ ♦❢ t❤❡ ♣❛rt✐❝❧❡ ♦♥ s♦♠❡

  t✐♠❡ ✐♥t❡r✈❛❧ [✵, T ] ❢r♦♠ t❤❡ ❢♦❝✉s ♦❢ t❤❡ ♠✐❝r♦s❝♦♣❡✳

  ❙❡❝♦♥❞ t❡r♠ ✿ ❛✈❡r❛❣❡ ♣♦✇❡r ✐♥ t❤❡ ❝♦♥tr♦❧ s✐❣♥❛❧✳ ❆✐♠ ✿ t♦ ♠✐♥✐♠✐③❡ t❤❡ ✜rst ❛♥❞ s❡❝♦♥❞ t❡r♠✳ t ❙t♦❝❤❛st✐❝ ❖♣t✐♠❛❧ ❈♦♥tr♦❧ Pr♦❜❧❡♠ ✿ t♦ ✜♥❞ t❤❡ ❢❡❡❞❜❛❝❦ str❛t❡❣② u

T [u]

  ✇❤✐❝❤ ♠✐♥✐♠✐③❡ t❤❡ ❝♦st ❢✉♥❝t✐♦♥❛❧ J ✳

  ❱❛r✐❛t✐♦♥ ✿ ♣r♦❜❧❡♠ ♦♥ ✐♥✜♥✐t❡ t✐♠❡ ❤♦r✐③♦♥ [✵, ∞)✿ Z T Z T ! !

  ✶ ✶

E E

✷ ✷ J dt u dt ∞ [u] = p (x t + z t ) + q .

  ❧✐♠ s✉♣ ❧✐♠ s✉♣ t T →∞ T →∞ T T

  ✵ ✵

  ❙t♦❝❤❛st✐❝ ❖♣t✐♠❛❧ ❈♦♥tr♦❧

  ❆ ❇r✐❡❢ ❍✐st♦r②

■♠♣❡t✉s ♦❢ ❖♣t✐♠❛❧ ❈♦♥tr♦❧ ✿ ❈❛❧❝✉❧✉s ♦❢ ❱❛r✐❛t✐♦♥ ✭✐♥ ✶✻✻✷ P✐❡rr❡ ❞❡ ❋❡r♠❛t

  ✉s❡❞ t❤❡ ♠❡t❤♦❞ ♦❢ ❝❛❧❝✉❧✉s t♦ ♠✐♥✐♠✐③❡ t❤❡ ♣❛ss❛❣❡ t✐♠❡ ♦❢ ❛ ❧✐❣❤t r❛② t❤r♦✉❣❤ t✇♦ ♦♣t✐❝❛❧ ♠❡❞✐❛✮ ❛♥❞ ❇r❛❝❤✐st♦❝❤r♦♥❡ ♣r♦❜❧❡♠ ✭✐♥ ✶✻✾✻ ❜② ❏♦❤❛♥ ❇❡r♥♦✉❧❧✐✮✳

  ❊❛r❧② ✇♦r❦ ♦♥ ♦♣t✐♠❛❧ ❝♦♥tr♦❧ t❤❡♦r② ✿ ✐♥ ✶✽✵✵s✱ ❡✳❣✳ ❍❛♠✐❧t♦♥✱ ❍✉r✇✐t③✱

  ▼❛①✇❡❧❧✱ P♦✐♥❝❛r❡✱ ▲②❛♣✉♥♦✈✱ ❲✐❡♥❡r✱ ❑♦❧♠♦❣♦r♦❢

  

▼♦❞❡r♥ ♦♣t✐♠❛❧ ❝♦♥tr♦❧ t❤❡♦r② ✿ ✐♥ t❤❡ ❡♥❞ ♦❢ t❤❡ ❲♦r❧❞ ❲❛r ■■✱ ❡✳❣✳ ❇❡❧❧♠❛♥✱

  ▲❛❙❛❧❧❡✱ ❇❧❛❝❦✇❡❧❧✱ ❋❧❡♠✐♥❣✱ ❇❡r❦♦✈✐t③✱ P♦♥tr②❛❣✐♥✱ ❑❛❧♠❛♥✳

  

❇❡❧❧♠❛♥✬s ❞②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣ ♠❡t❤♦❞ ✱ P♦♥tr②❛❣✐♥✬s ♠❛①✐♠✉♠ ♣r✐♥❝✐♣❧❡ ✱

❑❛❧♠❛♥✬s ▲◗ t❤❡♦r② ❙t♦❝❤❛st✐❝ ♦♣t✐♠❛❧ ❝♦♥tr♦❧ ✿

  ❇❡❧❧♠❛♥ ✐♥ ✶✾✺✷ ♠❡♥t✐♦♥❡❞ ✑st♦❝❤❛st✐❝ ❝♦♥tr♦❧✑ ✐♥ ♦♥❡ ♦❢ ❤✐s ❡❛r❧✐❡st ♣❛♣❡r✱ ❜✉t ■t♦✲t②♣❡ ❙❉❊ ✇❛s ♥♦t ✐♥✈♦❧✈❡❞✦ ❋❧♦r❡♥t✐♥ ✐♥ ✶✾✻✶ ❞❡r✐✈❡❞ ❢r♦♠ ❛ P❉❊ ❛ss♦❝✐❛t❡❞ ✇✐t❤ ❛ ❝♦♥tr♦❧❧❡❞ ▼❛r❦♦✈ ♣r♦❝❡ss ❜② ✉s✐♥❣ ❇❡❧❧♠❛♥✬s ❞②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣✳ ❑✉s❤♥❡r ✐♥ ✶✾✻✷ st✉❞✐❡❞ ♦♣t✐♠❛❧ ❝♦♥tr♦❧ ♣r♦❜❧❡♠ ✉s✐♥❣ ■t♦✲t②♣❡ ❙❉❊ ❛s t❤❡ st❛t❡ ❡q✉❛t✐♦♥✳ ▼❡rt♦♥ ✐♥ ❧❛t❡ ✶✾✻✵s✿ ✜rst ❛♣♣❧✐❝❛t✐♦♥ ♦❢ st♦❝❤❛st✐❝ ♦♣t✐♠❛❧ ❝♦♥tr♦❧ t❤❡♦r② t♦ ✜♥❛♥❝✐❛❧ ♠❛t❤❡♠❛t✐❝s✳

  ❙t♦❝❤❛st✐❝ ❖♣t✐♠❛❧ ❈♦♥tr♦❧

❙t♦❝❤❛st✐❝ ❖♣t✐♠❛❧ ❈♦♥tr♦❧ Pr♦❜❧❡♠ ✭❙❖❈P✮ ✿ ❝♦♠♣❧❡t❡❧② ♦❜s❡r✈❡❞ ❝♦♥tr♦❧

  ♣r♦❜❧❡♠ ✇✐t❤ ❛ st❛t❡ ❡q✉❛t✐♦♥ ♦❢ t❤❡ ■t♦ t②♣❡ ✭❞✐✛✉s✐♦♥ ♠♦❞❡❧✮ ❛♥❞ ✇✐t❤ ❛ ❝♦st ❢✉♥❝t✐♦♥❛❧ ♦❢ t❤❡ ❇♦❧③❛ t②♣❡✳ ❚❤❡ ❜❛s✐❝ s♦✉r❝❡ ♦❢ ✉♥❝❡rt❛✐♥t② ✐♥ ❞✐✛✉s✐♦♥ ♠♦❞❡❧s ✐s ✇❤✐t❡ ♥♦✐s❡ ✭t❤❡ t✐♠❡ ✑❞❡r✐✈❛t✐✈❡✑ ♦❢ t❤❡ ❇r♦✇♥✐❛♥ ♠♦t✐♦♥✮✱ ✇❤✐❝❤ r❡♣r❡s❡♥ts t❤❡ ❥♦✐♥t ❡✛❡❝ts ♦❢ ❛ ❧❛r❣❡ ♥✉♠❜❡r ♦❢ ✐♥❞❡♣❡♥❞❡♥t r❛♥❞♦♠ ❢♦r❝❡s ❛❝t✐♥❣ ♦♥ t❤❡ s②st❡♠s✳ ❚✇♦ ♠❛✐♥ ❛♣♣r♦❛❝❤❡s✿

  P♦♥tr②❛❣✐♥✬s ♠❛①✐♠✉♠ ♣r✐♥❝✐♣❧❡ ✿ ❍❛♠✐❧t♦♥✐❛♥ s②st❡♠ ✇❤✐❝❤ ❝♦♥s✐sts ♦❢ ❛❞❥♦✐♥t

❡q✉❛t✐♦♥ ✭❖❉❊✴❙❉❊✮✱ ♦r✐❣✐♥❛❧ st❛t❡ ❡q✉❛t✐♦♥ ❛♥❞ ♠❛①✐♠✉♠ ❝♦♥❞✐t✐♦♥s

❇❡❧❧♠❛♥✬s ❞②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣ ✿ s♦❧✈✐♥❣ ❍❏❇ ❡q✉❛t✐♦♥ ✇❤✐❝❤ ✐s ❛ P❉❊ ♦❢

✜rst ♦r❞❡r ✐♥ ❞❡t❡r♠✐♥✐st✐❝ ❝❛s❡ ❛♥❞ ♦❢ s❡❝♦♥❞ ♦r❞❡r ✐♥ st♦❝❤❛st✐❝ ❝❛s❡✳

  ❘❡❢❡r❡♥❝❡s ✿ ❲✳ ❍✳ ❋❧❡♠✐♥❣ ❛♥❞ ❍✳ ▼✳ ❙♦♥❡r✳ ✷✵✵✻✳ ✑❈♦♥tr♦❧❧❡❞ ▼❛r❦♦✈ Pr♦❝❡ss❡s ❛♥❞ ❱✐s❝♦s✐t② ❙♦❧✉t✐♦♥s✑✳ ❙♣r✐♥❣❡r✳ ❏✳ ❨♦♥❣ ❛♥❞ ❳✳ ❨✳ ❩❤♦✉✳ ✷✵✵✾✳ ✑❙t♦❝❤❛st✐❝ ❈♦♥tr♦❧s✳ ❍❛♠✐❧t♦♥✐❛♥ ❙②st❡♠s ❛♥❞ ❍❏❇ ❊q✉❛t✐♦♥s✑✳ ❙♣r✐♥❣❡r✳ ▼✳ ◆✐s✐♦✳ ✷✵✶✺✳ ✑❙t♦❝❤❛st✐❝ ❈♦♥tr♦❧ ❚❤❡♦r②✳ ❉②♥❛♠✐❝ Pr♦❣r❛♠♠✐♥❣

  Pr✐♥❝✐♣❧❡✑✳ ❙♣r✐♥❣❡r✳

  ❇❛s✐❝s ♦❢ ❙❖❈P

  ) , P ▲❡t Ω, F, (F t ❜❡ ❛ ✜❧t❡r❡❞ ♣r♦❜❛❜✐❧✐t② s♣❛❝❡ ♦♥ ✇❤✐❝❤ ❛♥ F t ✲❛❞❛♣t❡❞ t≥

  ✵

  ) ❇r♦✇♥✐❛♥ ♠♦t✐♦♥ B = (B t t≥ ✐s ❞❡✜♥❡❞✳ ❚❤❡ st❛t❡ ♦❢ t❤❡ ❝♦♥tr♦❧❧❡❞ s②st❡♠ ✐♥ ❛♥

  ✵

  ❙❖❈P ✐s ❞❡s❝r✐❜❡❞ ❜② ❛♥ ❙❉❊ u u u dX = b (t, X , u t ) dt + σ (t, X , u t ) dB t , t t t X = x.

n n n n×m

✵ ❍❡r❡ b : [✵, ∞) × R × U → R ❛♥❞ σ : [✵, ∞) × R × U → R ✱ ✇❤❡r❡ U ✐s t❤❡ q ❝♦♥tr♦❧ s❡t ✭t❤❡ s❡t ♦❢ ✈❛❧✉❡s t❤❛t t❤❡ ❝♦♥tr♦❧ ✐♥♣✉t ❝❛♥ t❛❦❡✮✳ ❖❢t❡♥✿ U = R ✳

  ❉❡✜♥✐t✐♦♥ t ) t≥

  ❚❤❡ ❝♦♥tr♦❧ str❛t❡❣② u = (u ✵ ✐s ❝❛❧❧❡❞ ❛♥ ❛❞♠✐ss✐❜❧❡ str❛t❡❣② ✐❢ u t t ✐s ❛♥ F ✲❛❞❛♣t❡❞ st♦❝❤❛st✐❝ ♣r♦❝❡ss u t (ω) ∈ U ❢♦r ❡✈❡r② (ω, t) ∈ Ω × [✵, ∞) u t❤❡ ❡q✉❛t✐♦♥ ❢♦r X t ❤❛s ❛ ✉♥✐q✉❡ str♦♥❣ s♦❧✉t✐♦♥✳ ▼♦r❡♦✈❡r✱ ❛♥ ❛❞♠✐ss✐❜❧❡ str❛t❡❣② u ✐s ❝❛❧❧❡❞ ❛ ▼❛r❦♦✈ str❛t❡❣② ✐❢ ✐t ✐s ♦❢ t❤❡ ❢♦r♠ u n u t = α (t, X ) t ❢♦r s♦♠❡ ❢✉♥❝t✐♦♥ α : [✵, ∞) × R → U✳

  ❚❤r❡❡ ❝♦♠♠♦♥ t②♣❡s ♦❢ ❝♦st ❢✉♥❝t✐♦♥❛❧s✿

  ❙❖❈P ♦♥ t❤❡ ✜♥✐t❡ t✐♠❡ ❤♦r✐③♦♥ [✵, T ] ✿ T !

  Z u u J w

  [u] = E (s, X , u s ) ds + z (X ) , s T n n ✵ ✇❤❡r❡ w : [✵, T ] × R × U → R ✭t❤❡ r✉♥♥✐♥❣ ❝♦st✮ ❛♥❞ z : R → R ✭t❤❡ t❡r♠✐♥❛❧ ❝♦st✮ ❛r❡ ♠❡❛s✉r❛❜❧❡ ❢✉♥❝t✐♦♥s ❛♥❞ T < ∞ ✐s t❤❡ t❡r♠✐♥❛❧ t✐♠❡✳

  ❙❖❈P ♦♥ ❛♥ ✐♥❞❡✜♥✐t❡ t✐♠❡ ❤♦r✐③♦♥ ✿ τ u !

  Z u u u J w

  [u] = E (X , u s ) ds + z (X ) , s τ n ✵ ✇❤❡r❡ S ⊂ R ✱ w : S × U → R✱ z : ∂S → R✱ ❛♥❞ t❤❡ st♦♣♣✐♥❣ t✐♠❡ u u τ = / ✐♥❢ {t > ✵ : X t ∈ S}

  ❙❖❈P ♦♥ ❛♥ ✐♥✜♥✐t❡ t✐♠❡ ❤♦r✐③♦♥ ✿ Z T ! E ✶ u u u

  J w [u] = (X , u s ) ds + z (X ) ,

  ❧✐♠ s✉♣ s τ T →∞ T n ✵ ✇❤❡r❡ w : R × U → R ✐s ♠❡❛s✉r❛❜❧❡✳

  ❇❡❧❧♠❛♥✬s ❞②♥❛♠✐❝ ♣r♦❣r❛♠♠✐♥❣ ♠❡t❤♦❞

  ●✐✈❡♥ t❤❡ st❛t❡ ♦❢ t❤❡ ❝♦♥tr♦❧❧❡❞ s②st❡♠ u u u dX

  X t t t ✵ = b (t, X , u t ) dt + σ (t, X , u t ) dB t , = x ✇✐t❤ ❛ ❝♦st ❢✉♥❝t✐♦♥❛❧ J[u]✳

  ❚❤❡ ❣♦❛❧ ✐s t♦ ✜♥❞

  u = α ♠✐♥ {J[u] : u ∈ U ✐s ❛ ▼❛r❦♦✈ str❛t❡❣② } ❉❡✜♥❡ t❤❡ ❣❡♥❡r❛t♦r L t ✱ α ∈ U ❛s n n m X X Xα ✶ i ik (t,x,α) jk ∂g ∂ g g b

  (x) = (t, x, α) (x) + σ σ (t, x, α) (x) L t i i j

  ∂x ∂x ∂x i = i ,j = k =

  ✶ ✶ ✶ u

  J ❛♥❞ t❤❡ ❝♦st✲t♦✲❣♦ ❢✉♥❝t✐♦♥ t ♦❢ t❤❡ ▼❛r❦♦✈ str❛t❡❣② u ❛s T ! u u u u Z

  J w t t s T F (X ) = E (s, X , u s ) ds + z (X ) t . t

  ❚❤❡♦r❡♠ t (x) ✶ ✷

  ❙✉♣♣♦s❡ t❤❡r❡ ✐s ❛ V ✱ ✇❤✐❝❤ ✐s C ✐♥ t ❛♥❞ C ✐♥ x✱ s✉❝❤ t❤❛t ∂V t (x) α

  • ∂t

  V V (x) = z(x), ♠✐♥ {L t (x) + w (t, x, α)} = ✵, T α∈U t

  (X ❛♥❞ |E (V ✵ ✵ )))| < ∞✱ ❛♥❞ ❝❤♦♦s❡ ❛ ♠✐♥✐♠✉♠ ∗ α

  V α t (t, x) ∈ ❛r❣♠✐♥ α∈U {L t (x) + w (t, x, α)} .

  ❉❡♥♦t❡ ❜② K t❤❡ ❝❧❛ss ♦❢ ❛❞♠✐ss✐❜❧❡ str❛t❡❣✐❡s u s✉❝❤ t❤❛t X n m X Z t ∂V s u ik u k i (X ) σ (s, X , u s ) dB s s s

  ∂x i = k = ✵ ✶ ✶ ∗ ∗ u t

  (t) = α , X ✐s ❛ ♠❛rt✐♥❣❛❧❡✱ ❛♥❞ s✉♣♣♦s❡ t❤❛t t❤❡ ❝♦♥tr♦❧ u t ❞❡✜♥❡s ❛♥ ❛❞♠✐ss✐❜❧❡ ▼❛r❦♦✈ str❛t❡❣② ✇❤✐❝❤ ✐s ✐♥ K✳ ❚❤❡♥✱ J[u ] ≤ J[u] ❢♦r ❛♥② u ∈ K✱ ❛♥❞ u

  V t (x) = J (x) t ✐s t❤❡ ✈❛❧✉❡ ❢✉♥❝t✐♦♥ ❢♦r t❤❡ ❝♦♥tr♦❧ ♣r♦❜❧❡♠✳

  ❙♦♠❡ ❊①❛♠♣❧❡s

  ❊①❛♠♣❧❡ ✶✿ ❚r❛❝❦✐♥❣ ❛ P❛rt✐❝❧❡

  ❙②st❡♠✿ dz t

  = βu , x = x + σB , t t t

  ✵

  dt z t ✐s t❤❡ ♣♦s✐t✐♦♥ ♦❢ t❤❡ s❧✐❞❡ r❡❧❛t✐✈❡ t♦ t❤❡ ❢♦❝✉s ♦❢ t❤❡ ♠✐❝r♦s❝♦♣❡ x t

  ✐s t❤❡ ♣♦s✐t✐♦♥ ♦❢ t❤❡ ♣❛rt✐❝❧❡ ✇❡ ✇✐s❤ t♦ ✈✐❡✇ ✉♥❞❡r t❤❡ ♠✐❝r♦s❝♦♣❡ r❡❧❛t✐✈❡ t♦ t❤❡ ❝❡♥t❡r ♦❢ t❤❡ s❧✐❞❡ β ∈ R ✐s t❤❡ ❣❛✐♥ ❝♦♥st❛♥t σ >

  ✵ ✐s t❤❡ ❞✐✛✉s✐♦♥ ❝♦♥st❛♥t ♦❢ t❤❡ ♣❛rt✐❝❧❡ t + z t ❆✐♠✿ t♦ ❦❡❡♣ t❤❡ ♣❛rt✐❝❧❡ ✐♥ ❢♦❝✉s ✭❦❡❡♣ x ❛s ❝❧♦s❡ t♦ ③❡r♦ ❛s ♣♦ss✐❜❧❡✮ ❛♥❞ t♦ ✐♥tr♦❞✉❝❡ ❛ ♣♦✇❡r ❝♦♥str❛✐♥t ♦♥ t❤❡ ❝♦♥tr♦❧ ❛s ✇❡❧❧✱ ❛s ✇❡ ❝❛♥♥♦t ❞r✐✈❡ t❤❡ s❡r✈♦ ♠♦t♦r ✇✐t❤ ❛r❜✐tr❛r✐❧② ❧❛r❣❡ ✐♥♣✉t ♣♦✇❡rs✳ ❈♦st ❋✉♥❝t✐♦♥❛❧✿ Z T Z T ! p q

  ✷ ✷

  J dt dt [u] = E (x t + z t ) (u t ) , +

  T T

  

✵ ✵

  ✇❤❡r❡ p, q > ✵ ❛❧❧♦✇ ✉s t♦ s❡❧❡❝t t❤❡ tr❛❞❡♦✛ ❜❡t✇❡❡♥ ❣♦♦❞ tr❛❝❦✐♥❣ ❛♥❞ ❧♦✇ ❢❡❡❞❜❛❝❦ ♣♦✇❡r✳

  ❊①❛♠♣❧❡ ✶✿ ❚r❛❝❦✐♥❣ ❛ P❛rt✐❝❧❡✳ ❝♦♥t✬❞✳

t + z t

  ❆s t❤❡ ❝♦♥tr♦❧ ❝♦st ❞❡♣❡♥❞s ♦♥❧② ♦♥ x ✱ ✐t ✐s ♠♦r❡ ❝♦♥✈❡♥✐❡♥t t♦ ♣r♦❝❡❡❞ t = x t + z t p q ❞✐r❡❝t❧② ✇✐t❤ t❤✐s q✉❛♥t✐t②✳ ❚❤❛t ✐s✱ ✇❡ ❞❡✜♥❡ e ✱ P = ❞❛♥ Q = T T ❛♥❞ ♥♦t❡ t❤❛t Z T Z T !

  ✷ ✷

  de t = βu t dt + σ dB t , J [u] = E P (x t + z t ) dt + Q (u t ) dt

  ✵ ✵

  ❲❡ ♦❜t❛✐♥ t❤❡ ❍❏❇ ❡q✉❛t✐♦♥✿

  ✷ ✷

  V ∂V t (x) σ ∂ t (x) ∂V t (x)

  ✷ ✷

  • α∈R

  ✷

  • β α + Px + Qα ✵ = ♠✐♥

  ∂t ∂x ∂x ✷

  ✷ ✷ ✷ ✷

  V ∂V t (x) σ ∂ t (x) β ∂V t (x)

  ✷

  = + + Px −

  ✷

  ∂t ∂x ∂x T (x) = ✷ ✹Q ✇✐t❤ V ✵ ✭❛s t❤❡r❡ ✐s ♥♦ t❡r♠✐♥❛❧ ❝♦st✮✱ ❛♥❞ ♠♦r❡♦✈❡r ∂V t (x) β ∂V t (x)

  ✷ α (t, x) = β α + Qα .

  ❛r❣♠✐♥ α∈R = − ∂x ∂x

  ✷Q

  ❊①❛♠♣❧❡ ✶✿ ❚r❛❝❦✐♥❣ ❛ P❛rt✐❝❧❡✳ ❝♦♥t✬❞✳ ✷ t (x) = a t + b t T (x) = x

  ❯s❡ ❛♥s❛t③ V ❛♥❞ V ✵✿

  ✷

  da t β db t

  ✷ ✷

  a a = T = + σ t = T = + P − t ✵, a ✵, ✵, b ✵. dt Q dt

  ❙♦❧✉t✐♦♥✿ r r ! !!

  ✷

  PQ P Q P σ a b t = β , t = β t❛♥❤ (T − t) ❧♥ ❝♦s❤ (T − t)

  ✷

  β Q β Q t (x) (t, x) ◆♦t❡ t❤❛t V ✐s s♠♦♦t❤ ✐♥ x ❛♥❞ t ❛♥❞ t❤❛t α ✐s ✉♥✐❢♦r♠❧② ▲✐♣s❝❤✐t③ ♦♥

  ✷

  [ + z ) ✵, T ]✳ ❍❡♥❝❡ ✐❢ ✇❡ ❛ss✉♠❡ t❤❛t E (x t t < ∞✱ t❤❡♥ ✇❡ ✜♥❞ t❤❛t t❤❡

  ❢❡❡❞❜❛❝❦ ❝♦♥tr♦❧ r r ! ∗ ∗ P P u t ) = − t❛♥❤ (T − t) = α (t, e t β (x t + z t ) ∗ ∗ Q Q s❛t✐s✜❡s u ∈ K✳ ❚❤✉s✱ ❜② t❤❡ ❛❜♦✈❡ t❤❡♦r❡♠ u ✐s ❛♥ ♦♣t✐♠❛❧ ❝♦♥tr♦❧ str❛t❡❣②✳ t t

  ❊①❛♠♣❧❡ ✷✿ ❖♣t✐♠❛❧ P♦rt❢♦❧✐♦ ❙❡❧❡❝t✐♦♥

  ❲❡ ❝♦♥s✐❞❡r✿ ❛ s✐♥❣❧❡ st♦❝❦ ✇✐t❤ ❛✈❡r❛❣❡ r❡t✉r♥ µ > ✵ ❛♥❞ ✈♦❧❛t✐❧✐t② σ > ✵ ❛ ❜❛♥❦ ❛❝❝♦✉♥t ✇✐t❤ ✐♥t❡r❡st r❛t❡ r > ✵✳

  ❉②♥❛♠✐❝ ♦❢ t❤❡ s②st❡♠ ✿

  dS = µS dt + σS dB , S = = rR dt , R = t t t t ✶, dR t t ✶.

  ✵ ✵ ❆ss✉♠♣t✐♦♥ ✿ ❲❡ ❝❛♥ ♠♦❞✐❢② ♦✉r ✐♥✈❡st♠❡♥t ❛t ❛♥② ♣♦✐♥t ✐♥ t✐♠❡✳

  ❲❡ ❝♦♥s✐❞❡r s❡❧❢✲✜♥❛♥❝✐♥❣ ✐♥✈❡st♠❡♥t str❛t❡❣✐❡s t t ▲❡t X ❜❡ t❤❡ t♦t❛❧ ✇❡❛❧t❤ ❛t t✐♠❡ t✱ ❛♥❞ ❜② u t❤❡ ❢r❛❝t✐♦♥ ♦❢ ♦✉r ✇❡❛❧t❤ t❤❛t ✐s ✐♥✈❡st❡❞ ✐♥ st♦❝❦ ❛t t✐♠❡ t✳ ❚❤❡♥ t❤❡ s❡❧❢✲✜♥❛♥❝✐♥❣ ❝♦♥❞✐t✐♦♥ ✐♠♣❧✐❡s t❤❛t dX dt t t + r ( t t + σu t t t . X dB

  = {µu ✶ − u )} X

  ●♦❛❧ ✿ ✭♦❜✈✐♦✉s❧②✮ t♦ ♠❛❦❡ ♠♦♥❡②✦

  ❊①❛♠♣❧❡ ✷✿ ❖♣t✐♠❛❧ P♦rt❢♦❧✐♦ ❙❡❧❡❝t✐♦♥✳ ❝♦♥t✬❞✳ t

  ▲❡t ✜① ❛ t❡r♠✐♥❛❧ t✐♠❡ T ✱ ❛♥❞ tr② t♦ ❝❤♦♦s❡ ❛ str❛t❡❣② u t❤❛t ♠❛①✐♠✐③❡s ❛ s✉✐t❛❜❧❡ ❢✉♥❝t✐♦♥❛❧ U ♦❢ ♦✉r t♦t❛❧ ✇❡❛❧t❤ ❛t t✐♠❡ T ❀ ✐✳❡✳ ✱ ✇❡ ❝❤♦♦s❡ t❤❡ ❝♦st ❢✉♥❝t✐♦♥❛❧ u

  J )) .

  [u] = E (−U (X T ❍♦✇ t♦ ❝❤♦♦s❡ ✉t✐❧✐t② ❢✉♥❝t✐♦♥❄

  ❚❤❡ ♦❜✈✐♦✉s ❝❤♦✐❝❡ U(x) = x t✉r♥s ♦✉t ♥♦t t♦ ❛❞♠✐t ❛♥ ♦♣t✐♠❛❧ ❝♦♥tr♦❧ ✐❢ ✇❡ s❡t U = R✱ ✇❤✐❧❡ ✐❢ ✇❡ s❡t U = [✵, ✶] ✭✇❡ ❞♦ ♥♦t ❛❧❧♦✇ ❜♦rr♦✇✐♥❣ ♠♦♥❡② ♦r s❡❧❧✐♥❣ s❤♦rt✮ t❤❡♥ ✇❡ ❣❡t ❛ r❛t❤❡r ❜♦r✐♥❣ ❛♥s✇❡r✿ ✇❡ s❤♦✉❧❞ ❛❧✇❛②s ♣✉t ❛❧❧ ♦✉r ♠♦♥❡② ✐♥ st♦❝❦ ✐❢ µ > r✱ ✇❤✐❧❡ ✐❢ µ ≤ r ✇❡ s❤♦✉❧❞ ♣✉t ❛❧❧ ♦✉r ♠♦♥❡② ✐♥ t❤❡ ❜❛♥❦✳ ❙✉♣♣♦s❡ t❤❛t U ✐s ♥♦♥❞❡❝r❡❛s✐♥❣ ❛♥❞ ❝♦♥❝❛✈❡✱ ❡✳❣✳✱ U(x) = ❧♥(x)✳ ❚❤❡♥ t❤❡ r❡❧❛t✐✈❡ ♣❡♥❛❧t② ❢♦r ❡♥❞✐♥❣ ✉♣ ✇✐t❤ ❛ ❧♦✇ t♦t❛❧ ✇❡❛❧t❤ ✐s ♠✉❝❤ ❤❡❛✈✐❡r t❤❛♥ ❢♦r U(x) = x✱ s♦ t❤❛t t❤❡ r❡s✉❧t✐♥❣ str❛t❡❣② ✇✐❧❧ ❜❡ ❧❡ss r✐s❦② ✭❝♦♥❝❛✈❡ ✉t✐❧✐t② ❢✉♥❝t✐♦♥s ❧❡❛❞ t♦ r✐s❦✲❛✈❡rs❡ str❛t❡❣✐❡s✱ ✇❤✐❧❡ t❤❡ ✉t✐❧✐t② U(x) = x ✐s ❝❛❧❧❡❞ r✐s❦✲♥❡✉tr❛❧✮✳ ❆s s✉❝❤✱ ✇❡ ✇♦✉❧❞ ❡①♣❡❝t t❤✐s ✐❞❡❛ t♦ t❡❧❧ ✉s t♦ ♣✉t s♦♠❡ ♠♦♥❡② ✐♥ t❤❡ ❜❛♥❦ t♦ r❡❞✉❝❡ ♦✉r r✐s❦✦

  ❊①❛♠♣❧❡ ✷✿ ❖♣t✐♠❛❧ P♦rt❢♦❧✐♦ ❙❡❧❡❝t✐♦♥✳ ❝♦♥t✬❞✳

  ❚❤❡ ❇❡❧❧♠❛♥ ❡q✉❛t✐♦♥ r❡❛❞s ✭✇✐t❤ U = R✮

  ✷ ✷ ✷ ✷

  ∂V t (x) σ α x ∂ V t (x) ∂V t (x)

  • α∈R
  • (µα + r ( ✵ = ♠✐♥ ✶ − α)) x

  

  ∂t ∂x ∂x ✷

  ✷

  ∂V t (x) ∂V t (x) (∂V t (x)/∂x) (µ − r)

  = + rx , −

  ✷ ✷ ✷

  V ∂t ∂x ∂ t (x)/∂x T ✷σ

  ✇❤❡r❡ V (x) = − ❧♥ x✱ ❛♥❞ ♠♦r❡♦✈❡r ∂V (x)/∂x µ − r t

  α , (t, x) = −

  ✷ ✷ ✷

  x

  V σ ∂ t (x)/∂x

  ✷ ✷

  V t (x)/∂x > ♣r♦✈✐❞❡❞ t❤❛t ∂ ✵ ❢♦r ❛❧❧ x > ✵✳ t t ❆♥s❛t③✿ V (x) = − ❧♥ x + b ✱ ❛♥❞

  ✷

  db t (µ − r) b T = . − C = ✵, ✵, C = r +

  ✷

  dt ✷σ

  ❊①❛♠♣❧❡ ✷✿ ❖♣t✐♠❛❧ P♦rt❢♦❧✐♦ ❙❡❧❡❝t✐♦♥✳ ❝♦♥t✬❞✳

  ❙♦❧✉t✐♦♥ t♦ ❇❡❧❧♠❛♥ ❡q✉❛t✐♦♥✿ V t

  (x) = − ❧♥ x − C (T − t)

  ✷ ✷

  V t (x)/∂x > ✭s♠♦♦t❤ ♦♥ x > ✵ ❛♥❞ ∂ ✵✮✳ ❚❤❡ ❝♦rr❡s♣♦♥❞✐♥❣ ❝♦♥tr♦❧✿ µ − r α (t, x) = .

  ✷

  σ ❇② ❡①✐st❡♥❝❡ t❤❡♦r❡♠ ♦❢ ❙❉❊✱ t❤❡ ❝♦♥❞✐t✐♦♥s ♦❢ ❛❜♦✈❡ t❤❡♦r❡♠ ❛r❡ ♠❡t ❛♥❞ ✇❡ ✜♥❞ t❤❛t t❤❡ ♦♣t✐♠❛❧ ❝♦♥tr♦❧✿

  µ − r u t = .

  ✷

  σ ❍❡♥❝❡✱ t❤❡ ❝❤♦✐❝❡ ♦❢ t❤❡ ✉t✐❧✐t② ❢✉♥❝t✐♦♥ t❡❧❧s ✉s t♦ ♣✉t ♠♦♥❡② ✐♥ t❤❡ ❜❛♥❦✱

  ✷

  ♣r♦✈✐❞❡❞ t❤❛t µ − r < σ ✳ ❖♥ t❤❡ ♦t❤❡r ❤❛♥❞✱ ✐❢ µ − r ✐s ❧❛r❣❡✱ ✐t ✐s ❜❡tt❡r t♦ ❜♦rr♦✇ ♠♦♥❡② ❢r♦♠ t❤❡ ❜❛♥❦ t♦ ✐♥✈❡st ✐♥ st♦❝❦ ✭t❤✐s ✐s ♣♦ss✐❜❧❡ ✐♥ t❤❡ ❝✉rr❡♥t s❡tt✐♥❣ ❛s ✇❡ ❤❛✈❡ ❝❤♦s❡♥ U = R✱ r❛t❤❡r t❤❛♥ r❡str✐❝t✐♥❣ t♦ U = [✵, ✶]✮✳

  ❖✉t❧♦♦❦✿ ❇❡②♦♥❞ ❇r♦✇♥✐❛♥ ▼♦t✐♦♥

  ■❞❡❛ ❢♦r ❣❡♥❡r❛❧✐③❛t✐♦♥s✴✐♠♣r♦✈❡♠❡♥ts✿ r❡♣❧❛❝❡ t❤❡ ❞r✐✈✐♥❣ ♣r♦❝❡ss✱ t ❇r♦✇♥✐❛♥ ♠♦t✐♦♥ B ✱ ✐♥ t❤❡ st❛t❡ ♦❢ ❝♦♥tr♦❧❧❡❞ s②st❡♠ ✇✐t❤ s♦♠❡ ♦t❤❡r t st♦❝❤❛st✐❝ ♣r♦❝❡ss M u u u dX M

  X t t t ✵ = b (t, X , u t ) dt + σ (t, X , u t ) d t , = x t ❈♦♠♠♦♥ ♦♣t✐♦♥s ❢♦r M ✿

  ▲é✈② ♣r♦❝❡ss ✿ ♣r♦❝❡ss ✇✐t❤ st❛t✐♦♥❛r② ❛♥❞ ✐♥❞❡♣❡♥❞❡♥t ✐♥❝r❡♠❡♥ts ✇✐t❤ ❝á❞❧á❣ s❛♠♣❧❡ ♣❛t❤s P✲❛✳s✳ ❋❡❛t✉r❡s✿ ✐♥✜♥✐t❡❧② ❞✐✈✐s✐❜❧❡ ❞✐str✐❜✉t❡❞✱ ▼❛r❦♦✈✱ s❡♠✐♠❛rt✐♥❣❛❧❡✱ ❥✉♠♣s ✐♥ t❤❡ tr❛❥❡❝t♦r✐❡s ❘❡❢❡r❡♥❝❡ ✿ ❇✳ ❖❦s❡♥❞❛❧ ❛♥❞ ❆✳ ❙✉❧❡♠✳ ✷✵✵✺✳ ✑❆♣♣❧✐❡❞ ❙t♦❝❤❛st✐❝ ❈♦♥tr♦❧ ♦❢ ❏✉♠♣ ❉✐✛✉s✐♦♥s✑✳ ❙♣r✐♥❣❡r , ❋r❛❝t✐♦♥❛❧ ❇r♦✇♥✐❛♥ ♠♦t✐♦♥ ✇✐t❤ ❍✉rst ♣❛r❛♠❡t❡r H ∈ (✵ ✶) ✐s ❛ ❝❡♥t❡r❡❞ H H t≥

  := (B t ) ●❛✉ss✐❛♥ ♣r♦❝❡ss B ✵ ✇✐t❤ ❝♦✈❛r✐❛♥❝❡ ❢✉♥❝t✐♦♥ H H E B t ✶ ✷H ✷H ✷H (B t s ) = + s − |t − s| ✳

  ✷ ❋❡❛t✉r❡s✿ ♥♦r♠❛❧❧② ❞✐str✐❜✉t❡❞✱ ❝♦♥t✐♥✉♦✉s tr❛❥❡❝t♦r✐❡s✱ ❧♦♥❣✴s❤♦rt ♠❡♠♦r②✱ ♥♦♥✲▼❛r❦♦✈ ✱ ♥♦♥✲♠❛rt✐♥❣❛❧❡

❘❡❢❡r❡♥❝❡ ✿ ❋✳ ❇✐❛❣✐♥✐ ❡t✳ ❛❧✳ ✷✵✵✽✳ ✑❙t♦❝❤❛st✐❝ ❈❛❧❝✉❧✉s ❢♦r ❋r❛❝t✐♦♥❛❧

❇r♦✇♥✐❛♥ ▼♦t✐♦♥ ❛♥❞ ❆♣♣❧✐❝❛t✐♦♥s✑✳ ❙♣r✐♥❣❡r

  

✿ ❇r♦✇♥✐❛♥ ♠♦t✐♦♥

✿ ❋r❛❝t✐♦♥❛❧ ❇r♦✇♥✐❛♥ ♠♦t✐♦♥ ✇✐t❤ ❞✐✛❡r❡♥t H

  

✿ P♦✐ss♦♥ ♣r♦❝❡ss

✿ ■♥❞❡♣❡♥❞❡♥t s✉♠ ♦❢ ❝♦♠♣♦✉♥❞ P♦✐ss♦♥ ♣r♦❝❡ss ✇✐t❤ ❇r♦✇♥✐❛♥ ♠♦t✐♦♥

  ▲é✈② ❉r✐✈❡♥ ❖♣t✐♠❛❧ ❈♦♥tr♦❧

  L = (L t ) t≥ t t ) , P

  

✵ ✐s ❛♥ F ✲❛❞❛♣t❡❞ ▲é✈② ♣r♦❝❡ss ❞❡✜♥❡❞ ♦♥ Ω, F, (F t≥ ✳

  ❚❤❡ st❛t❡ ♦❢ t❤❡ ❝♦♥tr♦❧❧❡❞ s②st❡♠✿ u u u u Z

  • dY = b (t, Y , u t ) dt + σ (t, Y , u t ) dB t γ (t, Y − , u , z) N (dt, dz) t t t t | {z } | {z } | {z } R n t

  ❞r✐❢t ♣❛rt ❞✐✛✉s✐♦♥ ♣❛rt n n n n n×m ❥✉♠♣ ♣❛rt ✇✐t❤ Y ✵ = y ∈ R ✱ b : [✵, ∞) × R × U → R ✱ σ : [✵, ∞) × R × U → R ✱ ❛♥❞ n n×m γ : [

  ✵, ∞) × R × U → R ✳ ❈♦st ❢✉♥❝t✐♦♥❛❧✿ τ

  Z u u J [u] = E w (X , u ) ds + z (X ) , s s τ nu

  / ✇❤❡r❡ S ⊂ R ✱ w : S × U → R✱ z : ∂S → R✱ ❛♥❞ τ = ✐♥❢ {t > ✵ : X t ∈ S}✳

  ❚❤❛♥❦ ②♦✉ ❢♦r ②♦✉r ❛tt❡♥t✐♦♥✦

Dokumen yang terkait

Dokumen baru