Two-Dimensional Probabilistic Risk Assessment Model 1525
Table VIII. Summary of the ANOVA Results for 200 Bootstrap Simulations
for F Value Statistics 95
Range Mean
Probability Frequency
c
Mean of
Factor
a
F Value
Range SD
Mean
b
Rank Rank
Temp1 1445
873, 2040 0.21
100 2.9
2–4 Temp2
2.7 0.3, 7.8
0.83 33
10.9 10–13
Temp3 1130
650, 1670 0.32
100 3.7
2–4 Time1
1550 1275, 1820
0.10 100
2.5 2–4
Time2 0.7
0.0, 3.6 1.75
2 12.5
10–13 Time3
6060 5250, 6860
0.05 100
1.0 1
d
MD 29
14, 45 0.41
100 8.9
8–9 LP1
273 236, 315
0.07 100
5.8 5–6
LP2 1.7
0.3, 4.3 0.68
22 11.4
10–13 LP3
325 295, 379
0.07 100
5.0 5–6
GT1 75
52, 97 0.26
100 7.0
7–8 GT2
1.9 0.3, 5.3
0.70 25
11.3 10–13
GT3 43
27, 55 0.29
100 8.1
7–9 Temp1 × Time1
1920 1660, 2150
0.07 100
Temp2 × Time2 0.7
0.3, 3.5 1.85
2 Temp3 × Time3
3605 3270, 4060
0.06 100
a
The abbreviations used for factors in this table are the same as those defined in Table I.
b
The ratio of standard deviation SD to mean F values for each factor also referred to as coefficient of variation.
c
The percentage of the bootstrap simulations for which the F values were statistically significant.
d
Time3 consistently had rank 1 in bootstrap simulations.
was specific to the example provided here and should not be used to make general quantitative judgments
regarding differences between F values obtained with different sample sizes or models. However, the tech-
nique used here can be applied to other case studies. The case study results suggest that the model output
has similar sensitivity to two or more factors if they have similar F values.
4.3.4. Summary of the Results for the Comingled Analysis of Variability and Uncertainty
Key insights and findings based on the comingled analysis of variability and uncertainty include:
r
Comingled analysis typically provided differ- ent results with respect to rank of impor-
tant inputs compared to the two-dimensional analysis. However, both probabilistic ap-
proaches identified similar groups of inputs with comparable importance.
r
Similar to the two-dimensional case, the two correlation-based methods failed to provide
the same insights with respect to identifica- tion of the most important inputs compared
to ANOVA.
r
In order to have statistically significantly differ- ent ranks, the two factors should have F values
that differ at least by 60 for the conditions of the case study.
4.4. Identifying Special Model Characteristics Using ANOVA
ANOVA provides insight regarding special model characteristics such as nonlinearity, thresholds
and saturation points, and interactions by compari- son of mean output values at different factor levels.
Table IX summarizes an example in which a set of contrasts was used to infer an interaction effect be-
tween storage temperature and storage time at Stage 3 and possible saturation points. When storage tem-
perature in Stage 3 was at the first level i.e., between 7.5
◦
C and 11
◦
C, storage time influenced growth of E. coli
in ground beef until the tenth day, since sta- tistically significant contrasts were estimated for the
difference in pathogen growth for consecutive days through the tenth. After the tenth day there was no
significant difference in the estimated growth, indicat- ing that maximum population density was achieved
by the tenth day. When storage temperature was at the second level i.e., between 11
◦
C and 14.5
◦
C, the
1526 Mokhtari and Frey
Table IX. Evaluation of ANOVA
Contrasts for the Growth Estimation Regarding the Interactions Between
Storage Temperature and Storage Time at Stage 3
Contrast Estimate
a
F Value
Pr F Significant
b
T [7.5–11
◦
C], Time 1st and 2nd days 0.005
112 0.0001
Yes T [7.5–11
◦
C], Time 2nd and 3rd days 0.026
1280 0.0001
Yes T [7.5–11
◦
C], Time 3rd and 4th days 0.049
1720 0.0001
Yes T [7.5–11
◦
C], Time 4th and 5th days 0.069
1280 0.0001
Yes T [7.5–11
◦
C], Time 5th and 6th days 0.074
610 0.0001
Yes T [7.5–11
◦
C], Time 6th and 7th days 0.103
455 0.0001
Yes T [7.5–11
◦
C], Time 7th and 8th days 0.042
30.8 0.0001
Yes T [7.5–11
◦
C], Time 8th and 9th days 0.031
6.8 0.008
Yes T [7.5–11
◦
C], Time 9th and 10th days 0.108
38.8 0.0001
Yes T [7.5–11
◦
C], Time 10th and 11th days ——
0.16 0.8
No T [11–14.5
◦
C], Time 1st and 2nd days 0.116
4060 0.0001
Yes T [11–14.5
◦
C], Time 2nd and 3rd days 0.211
5290 0.0001
Yes T [11–14.5
◦
C], Time 3rd and 4th days 0.218
2170 0.0001
Yes T [11–14.5
◦
C], Time 4th and 5th days 0.119
240 0.0001
Yes T [11–14.5
◦
C], Time 5th and 6th days 0.087
48.4 0.0001
Yes T [11–14.5
◦
C], Time 6th and 7th days ——
0.9 0.6
No T [18–21.5
◦
C], Time 1st and 2nd days 0.55
2630 0.0001
Yes T [18–21.5
◦
C], Time 2nd and 3rd days —–
0.1 0.4
No T [21.5–25
◦
C], Time 1st and 2nd days 0.503
6270 0.0001
Yes T [21.5–25
◦
C], Time 2nd and 3rd days —–
2.4 0.09
No
a
The Estimate column represents the estimate of the difference in the growth of the E. coli organisms in two consecutive days.
b
Contrasts with p values less than 0.05 are statistically significant.
contrasts indicated that maximum population den- sity was reached in only 5 days. When the storage
temperature increased to the third, fourth, and fifth levels, maximum population density was reached in
4, 3, and 2 days, respectively. This pattern indicated an interaction effect between storage time and tem-
perature and the corresponding saturation points for growth.
Proper definition of factor levels for storage time or storage temperature can help identify thresholds
with respect to these factors, which can be directly used as CLs in risk management recommendations.
For example, knowing that growth is zero or negligible in the beginning of the storage process, one may look
for two consecutive time intervals in which growth starts to show a statistically significant increase, which
indicates a threshold. Using the inferred threshold as a recommended storage time can ensure prevention
of growth of the E. coli O157:H7 organisms in ground beef servings.
4.5. Verification of the ANOVA Results
