10a SOC 681 How to Fool Yourself with SEM
How to Fool
Yourself with SEM
James G. Anderson, Ph.D
Purdue University
Tripping at the Starting Line:
Specification
1.
2.
3.
4.
Specify the model after the data are
collected rather than before.
Omit causes that are correlated with other
variables in a structural model.
Fail to have sufficient numbers of indicator
latent variables.
Use psychometrically inadequate
measures.
Tripping at the Starting Line:
Specification (2)
5.
6.
7.
Fail to give careful consideration to the
question of directionality.
Specify feedback effects in structural
models as a way to mask uncertainty
about directionality.
Overfit the model (e.g., forget the goal of
parsimony.
Tripping at the Starting Line:
Specification (3)
8.
9.
Add disturbance or measurement error
correlations without substantive reason.
Specify that indicators load on more than
one factor without substantive reason.
Improper Care and Feeding: Data
10. Don’t check the accuracy of data input or
coding.
11. Ignore whether the pattern of data loss is
random or systematic.
12. Fail to examine distributional characteristics.
13. Don’t screen for outliers.
14. Assume that all relations are linear.
Checking Critical Judgment at the Door:
Analysis and Re-specification
15.
16.
17.
18.
Re-specify a model based entirely upon
statistical criteria.
Fail to check the accuracy of your
programming.
Analyze a correlation matrix when it is
inappropriate.
Analyze variables so highly correlated that
the solution is unstable.
Checking Critical Judgment at the Door:
Analysis and Re-specification (2)
19.
20.
21.
22.
Estimate a very complex model with a
small sample.
Set scales for latent variables
inappropriately.
Ignore the problem of starting values or
provide grossly inaccurate ones.
When identification status is uncertain, fail
to conduct tests of solution uniqueness.
Checking Critical Judgment at the Door:
Analysis and Re-specification (3)
23.Fail to recognize empirical underidentification.
24. Fail to separately evaluate the
measurement and structural portions
a hybrid model.
of
The Garden Path: Interpretation
25.
26.
27.
28.
Look only at indices of overall fit and ignore
other types of fit information.
Interpret goodness-of-fit as meaning that
the model is “proved”.
Interpret goodness-of-fit as meaning that
the endogenous variables are strongly
predicted.
Rely too much upon significance tests.
The Garden Path: Interpretation (2)
29.
30.
31.
32.
Interpret the standardized solution in
inappropriate ways.
Fail to consider equivalent models.
Fail to consider alternative models.
Reify Factors
The Garden Path: Interpretation(3)
33. Believe that a strong analytical method like SEM
can compensate for poor study design or poor
ideas.
34. As the researcher, fail to report enough
information so that your readers can reproduce
your results.
35. Interpret estimates of large direct effects from a
structural model as “proof” of causality.
Reference
R.B. Kline, Principles and Practice of
Structural Equation Modeling, NY:
Guilford Press, 1998.
Yourself with SEM
James G. Anderson, Ph.D
Purdue University
Tripping at the Starting Line:
Specification
1.
2.
3.
4.
Specify the model after the data are
collected rather than before.
Omit causes that are correlated with other
variables in a structural model.
Fail to have sufficient numbers of indicator
latent variables.
Use psychometrically inadequate
measures.
Tripping at the Starting Line:
Specification (2)
5.
6.
7.
Fail to give careful consideration to the
question of directionality.
Specify feedback effects in structural
models as a way to mask uncertainty
about directionality.
Overfit the model (e.g., forget the goal of
parsimony.
Tripping at the Starting Line:
Specification (3)
8.
9.
Add disturbance or measurement error
correlations without substantive reason.
Specify that indicators load on more than
one factor without substantive reason.
Improper Care and Feeding: Data
10. Don’t check the accuracy of data input or
coding.
11. Ignore whether the pattern of data loss is
random or systematic.
12. Fail to examine distributional characteristics.
13. Don’t screen for outliers.
14. Assume that all relations are linear.
Checking Critical Judgment at the Door:
Analysis and Re-specification
15.
16.
17.
18.
Re-specify a model based entirely upon
statistical criteria.
Fail to check the accuracy of your
programming.
Analyze a correlation matrix when it is
inappropriate.
Analyze variables so highly correlated that
the solution is unstable.
Checking Critical Judgment at the Door:
Analysis and Re-specification (2)
19.
20.
21.
22.
Estimate a very complex model with a
small sample.
Set scales for latent variables
inappropriately.
Ignore the problem of starting values or
provide grossly inaccurate ones.
When identification status is uncertain, fail
to conduct tests of solution uniqueness.
Checking Critical Judgment at the Door:
Analysis and Re-specification (3)
23.Fail to recognize empirical underidentification.
24. Fail to separately evaluate the
measurement and structural portions
a hybrid model.
of
The Garden Path: Interpretation
25.
26.
27.
28.
Look only at indices of overall fit and ignore
other types of fit information.
Interpret goodness-of-fit as meaning that
the model is “proved”.
Interpret goodness-of-fit as meaning that
the endogenous variables are strongly
predicted.
Rely too much upon significance tests.
The Garden Path: Interpretation (2)
29.
30.
31.
32.
Interpret the standardized solution in
inappropriate ways.
Fail to consider equivalent models.
Fail to consider alternative models.
Reify Factors
The Garden Path: Interpretation(3)
33. Believe that a strong analytical method like SEM
can compensate for poor study design or poor
ideas.
34. As the researcher, fail to report enough
information so that your readers can reproduce
your results.
35. Interpret estimates of large direct effects from a
structural model as “proof” of causality.
Reference
R.B. Kline, Principles and Practice of
Structural Equation Modeling, NY:
Guilford Press, 1998.