1516 Mokhtari and Frey
Table II. ANOVA Table for a Two-Factor Model
Source of Variation Sum of Square
a
Degree of Freedom
b
Mean Square F
Value
c
Factor A SSA = nb
¯ Y
i..
− ¯ Y
... 2
a − 1
MSA = SSA
a − 1
F
A
= MSA
MSE Factor B
SSB = na ¯
Y
. j.
− ¯ Y
... 2
b − 1
MSB = SSB
b − 1
F
B
= MSB
MSE Interaction
SSAB = n ¯
Y
i j.
− ¯ Y
i..
− ¯ Y
. j.
+ ¯ Y
... 2
a − 1 × b − 1 MSAB =
SSAB a − 1 × b − 1
F
AB
= MSAB
MSE Error
SSE = Y
i j k
− ¯ Y
ij. 2
a × b × n − 1 MSE =
SSE a × b ×
n − 1 Total
SSTO = Y
ijk
− ¯ Y
... 2
n × a × b − 1
a
¯ Y
i..
=
b j =
1 n
k= 1
Y
i j k
b × n , ¯
Y
. j.
=
a i =
1 n
k= 1
Y
i j k
a × n , ¯
Y
...
=
a i =
1 b
j = 1
n k=
1
Y
i j k
a × b × n .
b
a = number of levels for Factor A; b = number of levels for Factor B; n = number of values for the response variable.
c
5 significance level is considered for statistically significant F values.
as a useful benchmark for comparison with ANOVA. The Pearson correlation coefficient PCC, for in-
stance, can be used to characterize the degree of linear relationship between the output values and sampled
values of individual inputs. If the relationship between an input and an output is nonlinear but monotonic,
Spearman correlation coefficients SCC provide bet- ter performance.
38–40
SCCs are based on ranks, not sample values, of each input and output. Neither PCCs
nor SCCs can provide insight regarding possible in- teraction effects between inputs. The magnitude of a
PCC or SCC is typically used as a basis to rank order model inputs based on their influence on the output.
3.3. Top-Down Correlation for Comparison of Sensitivity Analysis Methods
When applying different sensitivity analysis methods to a case study, an important question is
whether different techniques agree in their identifica- tion of important inputs. The so-called top-down cor-
relation method is useful for this purpose. Details of the method are available elsewhere.
36,41
This method gives greater weight to agreement or disagreement in
rank ordering of the most important inputs and gives less weight to comparisons in ranks of inputs with low
importance. Large positive values for the top-down correlation result indicate agreement between two
sets of ranks for the most important inputs.
3.4. Probabilistic Analysis Scenarios
Two scenarios are introduced: 1 variability anal- ysis for different realizations of uncertainty; and 2
comingled analysis of variability and uncertainty. Pro- cedures for application of sensitivity analysis to these
scenarios and specific insights with respect to sensitiv- ity that can be obtained from each of these approaches
are briefly discussed.
3.4.1. Variability Analysis for Different Uncertainty Realizations
In variability analysis for different uncertainty re- alizations, the objective of the analysis is to distinguish
between variability and uncertainty in order to prop- erly distinguish between inherent differences in val-
ues among members of a population versus lack of knowledge.
2,15,19
As illustrated in Fig. 2, the focus of sensitivity analysis in this approach is to identify
the key variability inputs for each realization of un- certainty.
1,2
A realization refers to one model simu- lation based upon one randomly sampled uncertainty
value for each probabilistic input. In this situation, sensitivity analysis provides insight regarding whether
the identification of the key sources of variability is robust with respect to uncertainty. Each data set in
Fig. 2 includes randomly generated values e.g., m values from variability distributions of each model
input for a given uncertainty realization and the cor- responding model output values. Sensitivity analysis
is applied separately to each data set. Thus, for each realization, the key sources of variability, critical lim-
its, or both, are identified. This process is repeated n
times to arrive at different sensitivity rankings for variability inputs, where n refers to the number of un-
certainty realizations. A ranking represents the com- parative order of importance of each input for a given
Two-Dimensional Probabilistic Risk Assessment Model 1517
Fig. 2. Case study scenario for application of sensitivity analysis to
variability analysis for different uncertainty realizations of a model.
uncertainty realization of the model when the inputs are sorted according to their sensitivity indices. A rank
of 1 is assigned to an input with the highest sensi- tivity index, and the largest numerical value of rank
was assigned to the input with the least importance i.e., lowest sensitivity index. For example, when us-
ing ANOVA, for each uncertainty realization inputs are ranked based on the relative magnitude of F val-
ues, and hence, n ranks are assigned to each input, where n represents the number of uncertainty real-
izations of the model. For this probabilistic scenario, values of 100 and 650 were selected for n and m, re-
spectively, based upon the storage limit in each Excel sheet.
To the extent that the sensitivity analyses yield similar results about the rank ordering of key in-
puts regardless of uncertainty, an analyst or deci- sionmaker will have greater confidence that the re-
sults of the analysis are robust to uncertainty. If the ranking of key inputs changes substantially from
one realization of uncertainty to another, the iden- tification of key inputs would be uncertain. Addi-
tional data collection or research may reduce this ambiguity.
3.4.2. Comingled One-Dimensional Variability and Uncertainty Analysis
There may be situations in which it is either not necessary or perhaps not possible to distinguish be-
tween variability and uncertainty. If either variability or uncertainty dominates the assessment, then it may
not be necessary to distinguish between the two, nor is it necessary to separate the two if the focus is on
a randomly selected individual. Furthermore, during the process of model building, an analyst may want
to estimate the widest range of values that might be assigned to each model input for purposes of verify-
ing the model and evaluating the robustness of the model to large perturbations in its inputs. As a pre-
liminary step in prioritizing data collection or the de- velopment of distributions for model inputs, an an-
alyst may wish to assess the key sources of variation regardless of whether they represent variability or un-
certainty. In some cases, an analyst may make a judg- ment that it is difficult to separate variability from
uncertainty,
6
or that a two-dimensional probabilis- tic approach is impractical or not necessary based
upon the assessment objectives. Therefore, results
1518 Mokhtari and Frey
Fig. 3. Case study scenario for application of sensitivity analysis to the
comingled variability and uncertainty analysis of a model.
based on a “one dimensional” comingling of vari- ability and uncertainty are compared to those from
a “two-dimensional” simulation in which variability and uncertainty are distinguished. The latter is typi-
cally preferred where possible and appropriate to the assessment objective.
Fig. 3 illustrates sensitivity analysis in the context of comingled variability and uncertainty. There is one
data set generated for each of the uncertainty real- izations of the model. Each data set has m rows of
randomly generated samples from variability distri- butions and the corresponding output values. Prior
to application of sensitivity analysis, these data sets are appended together to form a single data set with
n × m
rows. Sensitivity analysis is then applied to the comingled data set and a set of rankings is obtained.
For the comingled analysis in the growth estimation part values of 100 and 650 were selected for n and m,
respectively.
4. RESULTS
