Penguk ur an K i ner j a d an Est im asi

11.3 Penguk ur an K i ner j a d an Est im asi

Consider a set of out put values for t he same measure Y 1 ;Y 2 ;:::;Y n (e.g. delays of n di¤erent runs, or wait ing t imes of n di¤erent runs). We want t o have

² a point est imat e t o approximat e t he t rue value of Y i , and ² an int er val est imat e t o out line t he range where t he t rue value lies.

11.3.1 Est i m asi T i t i k

² The point est imat or of based on t he dat a Y 1 ;Y 2 ;:::;Y n is de… ned by

b 1 µ= X Y i

n i=1

² The point est imat or b µ is said t o be unbiased for µ if

µ) = µ E (b

I n general

µ) = µ + b E (b

i.e. t here is a drift ing, or bias. ² For cont inuous dat a, t he point est imat or Á of based on dat a f Y (t ); 0 ·

t·T E g, where T E is t he simulat ion run lengt h, is de… ned by

b Á=

Y (t )dt

and is called a t ime average of Y(t ) over [0; T E ].

² I n general

E(b Á) = Á+ b

if b = 0, b Á is said t o be unbiased for Á ² One performance measure of t hese est imat ors (point or int erval) is a

quant ile or a percent ile. – Quant iles describe t he level of performance t hat can be delivered

wit h a given pr obability p – Assume Y represent s t he delay in queue a cust omer experiences,

t hen t he 0.85 quant ile (or 85% per cent ile) of Y is t he value ° such t hat

P(Y · ° ) = p = 0:85

11.3.2 Est i m asi I nt er val

² Valid int erval est imat ion t ypically requires a met hod of est imat ing t he variance of t he point est imat or b Á or b µ.

² Let ¾ 2 (b µ) = var (b µ) represent t he t rue variance of a point est imat or

b µ, and let b 2 ¾ (b µ) represent an est imat or of ¾ 2 (b µ) based on t he dat a

² Suppose t hat

E 2 b ¾ (b µ) =B¾ 2 (b µ)

where B is called t he bias in t he variance est imat or. ² I t is desirable t o have

B=1

in which case b 2 ¾ (b µ) is said t o be an unbiased est imat or of variance, ¾ 2 (b µ).

² I f it is an unbiased est imat or, t he st at ist ic

b µ¡µ t=

b ¾(b µ)

is approximat ly t dist ribut ion wit h some degree f of freedom. ² A n approximat e 100(1 ¡ ®)% con…dence int erval for µ is given by

b µ¡t ®=2;f ¾(b b µ) · µ · b µ+t ®=2;f b ¾(b µ) ² This relat ion involves t hree paramet ers, est imat or for mean, est imat or

for variance, and t he degree of freedom. How t o det er minet hese values? – Est imat or for mean is calculat ed as above as a point est imat or

b µ=

i= 1

– Est imat or for t he variance and for t he degree of freedom has t o consider t wo separat e cases

1. If Y i s are st at ist ically independent observat ions t hen use ¤

X n Y i ¡b µ S 2 = n¡1

i=1

1. t o calculat e

b ¾ (b µ) =

1. If Y i s ar enot st at ist ically independent , t hen t he above est ima- t or for variance is biased. Y i s is an aut ocorrelat ed sequence, somet imes called a t ime series. In t his case,

(b µ) = var (b µ) = 2 cov (Y i ;Y j )

i=1 j=1

i.e. one needs t o calculat e co-variance for every possible pair of observat ions. Too expensive. If t he simulat ion is long eough t o have passed t he t ransient phase, t heout put is approximat ely covar iance stationary. That is Y i+k depends on Y i+1 in t he same way as Y k depends on Y 1 For a covariance st at ionary t ime series s, de… ne t he lag k aut ocovariance by

° k = cov(Y 1 ;Y 1+ k ) = cov(Y i ;Y i+k ) For k = 0, ° 0 becomes t he populat ion variance ° 0 = cov(Y 0+ i ;Y i ) = var (Y i ) The lag k aut ocorrelat ion is t he correlat ion bet ween any t wo

observat ions k apart .

and has t he pr opert y ¡1·½ k · 1; k = 1; 2; : : : If a t ime series is covariance st at ionary, t hen t he calculat ion

of sample variance can be subst ant ially simpli…ed.

So all we need is t o calculat e covariance bet ween one sample and every ot her samples, but not every sample wit h every ot her samples.