OUTPUT ANALYSIS VIA THE ARENA OUTPUT ANALYZER
9.6 OUTPUT ANALYSIS VIA THE ARENA OUTPUT ANALYZER
The Arena Output Analyzer is a tool that supports statistical analysis of replication output data (output analysis). Such data are collected and stored in data files during a simulation run in accordance with any statistical element defined in the Statistic module or in Record modules (see Sections 5.4 and 5.5). The Output Analyzer then provides
Output Analysis 183 options to manipulate, analyze, and display the data and related statistics. The options
include the following: Data transfer (e.g., import/export of ASCII files)
Statistical analysis (batching, correlogram, point estimation, and confidence interval estimation for means and standard deviations, and statistical tests for comparing parameters of different samples) Graphing data and statistics (plots and charts)
The reader is reminded at this juncture that the working example of Section 9.5 will also be used throughout this section.
9.6.1 D ATA C OLLECTION
During each replication, observations are collected and written continually to data files. This mechanism preserves a record of the observations for later inspection or for statistical analysis. For instance, file G 1 Delay contains buffer delays for parts of type
G 1. Using the Output Analyzer, the analyst can obtain a human-readable version of the file by using the Export option from the Data File item of the File pull-down menu. As an example, consider Table 9.4, which displays buffer delays of type G 1 parts, recorded during the first 50 hours of a replication.
Table 9.4 Tallied buffer delay observations for parts of type G 1
Simulation Collection Time Observed Time in Queue G 1 Q Time
0:00000000e þ 000 0:00000000e þ 000 2:40000000e þ 000
1:70769546e þ 000 3:40000000e þ 000
2:16722249e þ 000 4:40000000e þ 000
3:04687979e þ 000 5:40000000e þ 000
3:76552538e þ 000 7:80000000e þ 000
5:91574045e þ 000 8:80000000e þ 000
5:33219193e þ 000 9:80000000e þ 000
2:49312870e þ 000 1:09922681e þ 001
0:00000000e þ 000 1:19922681e þ 001
7:17802770e 001 1:47079443e þ 001
0:00000000e þ 000 1:57079443e þ 001
7:57823257e 001 1:67079443e þ 001
4:95948674e 001 1:77505471e þ 001
0:00000000e þ 000 1:87505471e þ 001
2:95582065e 001 1:97505471e þ 001
8:78091690e 001 2:07505471e þ 001
1:47883109e þ 000 2:70131256e þ 001
4:35622213e 001 3:09594790e þ 001
0:00000000e þ 000 4:28340274e þ 001
0:00000000e þ 000 4:55615801e þ 001
0:00000000e þ 000 4:65615801e þ 001
4:35670033e 001 1:00000000e þ 000
1:00000000e þ 000
184 Output Analysis The exported data are arranged in two columns, so that each row entry consists of
a pair of values. The first value (column 1) is the simulation time of the collected observation (time stamp), while the second value (column 2) is the observed value at that time. Note that time 1 :0 in the last line of Table 9.4 stands for EOF (end of file). In terms of Arena variables, every part departure immediately triggers the recording of the corresponding pair of values as a row in the output file.
9.6.2 G RAPHICAL S TATISTICS
Figure 9.10 displays statistics of type G 1 parts. This display consists of four panels, each depicting graphical statistics for buffer delays of type G 1 parts, collected over 1000 simulation hours. Starting at the upper left corner and proceeding clockwise, the graphs display the following:
A plot (continuous-curve graph) of the sequence of delay times recorded
A bar chart of the same
A histogram of the collected delay times and its cdf The moving average of the collected delay times (legend label Smoothed), superim- posed on the actual delay times (legend label Raw Data)
An examination of the histogram panel reveals some probability mass in the histo- gram tail. This is due to the bursty nature of Poisson arrivals: When a burst of parts arrives (this can be visualized as a cluster of arrivals on the timeline), then burst
Figure 9.10 Graphical statistics for parts of type G 1.
Output Analysis 185 members at its tail end incur higher-than-average delay times. These unusually long
delays leave their mark in all four panels. For example, the plot, bar chart, and moving average panels exhibit sharp upward fluctuations corresponding to burst arrivals, while the histogram panel quantifies this effect in the shape of its tail.
9.6.3 B ATCHING D ATA FOR I NDEPENDENT O BSERVATIONS
The Output Analyzer provides a number of statistical analysis tools as part of the Analyze pull-down menu. For example, using the Batch/Truncate Obs'ns option in the Analyze menu, the user can batch observations in the style of the batch means method to estimate confidence intervals without relying on the Arena standard output. As usual, batching implementation depends on the type of statistics observed: discrete sample (observation based) or continuous sample (time based) as described in Section 9.2.2. For instance, suppose we wish to construct a 95% confidence interval for the mean buffer delay of type G 1 parts using 500,246 observations over a replication length of 1,000,000 hours. Figure 9.11 displays the requisite Batch/Truncate dialog box for observations grouped into batches of size 1000.
For every batching action, the Arena Output Analyzer produces a summary report that displays the batching plan parameters and the resultant estimated batch covariance. In particular, Panel 9.1 displays the batching results specified in Figure 9.11.
Figure 9.11 Batch/Truncate dialog box for delay times of G_1 type parts.
186 Output Analysis
Batch/Truncate Summary G1 Delay Batched Batched observations stored in file :
G1_Delay_Batched.flt Initial Observations Truncated :
500 Number of Observations Per Batch :
Number of Batches :
1000 Number of Trailing Obs’ns Truncated :
246 Estimate of Covariance Between Batches :
Panel 9.1 Batch/Truncate summary report for delay times of G 1 type parts.
The report in Panel 9.1 shows that we chose not to truncate any observations from the transient period. All in all, 500 batches of 1000 observations each were generated with 246 trailing observations (for the last incomplete batch). As a check on whether
a batch size of 1000 suffices to yield approximately statistically independent batches, the report also provides an estimate of the covariance between batches. (Recall from Section 3.6 that covariance or correlation values close to 0 suggest that batches are uncorrelated, and consequently, approximately independent. The lack of correlation suggests, but does not imply, statistical independence.) Arena actually performs a statistical test (see Section 3.10) to determine whether the null hypothesis that the covariance is 0 can be rejected at a given significance level a. In our case, the cov-
ariance estimate is 0 :03458, which is sufficiently close to zero to conclude that the null hypothesis cannot be rejected at significance level a ¼ 0:05. If, however, Arena found that the null hypothesis must be rejected, it would then issue a message such as
Covariance equal to 0 rejected in favor of Covariance > 0 at 0:05 level: Such a message indicates that the batching plan should be modified. Generally, increasing
the batch size should reduce the absolute value of batch covariance (or correlation).
9.6.4 C ONFIDENCE I NTERVALS FOR M EANS AND V ARIANCES
Having verified that the batching plan appears statistically valid, we may now proceed to estimate the requisite confidence interval using the estimates obtained. To this end, the analyst may use the Conf. Interval On Mean option in the Analyze menu, and then select among two methods for computing confidence intervals for means:
Classical or Standardized Time Series. . . Figure 9.12 displays 95% classical confidence intervals for mean buffer delays for parts of type G 1 and type G 2, separately. Observe that these confidence intervals are considerably tighter than those in Table 9.3. Again, if more than one replication is available, then the user may elect to pool all the mean buffer delay estimates into a single sample, and use it to compute generally tighter confidence intervals. This can be achieved by selecting the Lumped option in the Replications field of the Data File dialog box.
Output Analysis 187
Figure 9.12 Confidence intervals for mean delays of parts of type G 1 and type G 2.
In a similar vein, the analyst can produce confidence intervals for the standard deviation, using the Conf. Interval on Std Dev. . . option in the Analyze menu.
9.6.5 C OMPARING M EANS AND V ARIANCES
The Analyze menu also provides the options Compare Means and Compare Vari- ances for comparing the means and variances, respectively, of two samples drawn from two populations, by testing them statistically for equality. For example, to test the null hypothesis that the means are equal, Arena sets up a confidence interval for their difference. Since under the null hypothesis the difference is zero, one accepts equality if the confidence interval includes 0, and rejects it, otherwise.
Figure 9.13 depicts the dialog boxes for comparing the mean delays of the two part types, G 1 and G 2, from their data files of observations. The test results are shown in Figure 9.14 via a confidence interval for the difference of the respective mean delays, at significance level a ¼ 0:05.
An examination of the graphic at the top as well as the numerical information at the bottom of Figure 9.14 reveals that the confidence interval for the difference does indeed include 0. Consequently, the null hypothesis of means’ equality cannot be rejected at that significance level. This result is in line with expectations stemming from theoretical considerations. Indeed, queueing theory tells us that in a queueing system with multiple equal-priority job classes, a FIFO discipline, and a single server, the mean delay time of all job classes is the same. Of course, the average numbers of type G 1 and G 2 parts in the buffer are different due to their disparate arrival and service processes.
188 Output Analysis
Figure 9.13 Dialog boxes for comparing mean buffer delays of type G 1 and type G 2 parts.
In a similar vein, the variances of two samples drawn from two populations can also
be compared for equality, using the Compare Variances option of the Analyze menu. Again, Arena performs a statistical test of the null hypothesis that the true variances are equal by constructing a confidence interval, this time for the ratio of the sample
Figure 9.14 Test results for the equality of mean buffer delays of type G 1 and type G 2 parts.
Output Analysis 189 variances of the two samples. Since under the null hypothesis the ratio is 1, one accepts
the null hypothesis if the confidence interval includes 1, and rejects it, otherwise.
9.6.6 P OINT E STIMATES FOR C ORRELATIONS
To gauge the statistical dependence among observations within a sample, the analyst can make use of the Correlogram option in the Analyze menu. This option computes the sample autocorrelation function (see Section 3.9), and displays the numerical values of the autocorrelation point estimates, as well as their graph, as function of the lag. (Recall that the value of a correlation coefficient ranges between
1 and þ1.) Figure 9.15 displays two correlograms for delays of type G 1 parts, based on the same underlying sample. The left-side correlogram was computed from simulation observations of successive delays. The correlogram at the right was computed from the sample means of batches of 1000 delays. The two correlograms are strikingly different: The successive delays are strongly positively correlated, while the sample means of the batches are very nearly uncorrelated. This example illustrates how a good batching plan can largely eliminate correlations among sample mean estimates, resulting in confidence intervals for the sample parameters that legitimately invoke the central limit theorem (see Section 3.8.5).
Figure 9.15 Results of correlation analysis for type G 1 parts.
190 Output Analysis