06 SOC 681 Multiple Sample Models
Multiple Sample Models
James G. Anderson, Ph.D.
Purdue University
Rationale of Multiple
Sample SEMs
• Do estimates of model parameters vary
across groups?
• Another way of asking this question is:
Does group membership moderate the
relationships specified in the model?
Uses of Multiple Sample SEMs
•
•
Use for analysis of cross-sectional,
longitudinal, experimental and quasiexperimental data and to test for measurement
variance.
This procedure allows the investigator to:
1) Estimate separately the parameters for multiple
samples
2) Test whether specified parameters are equivalent
across these groups.
3) Test whether there are group mean differences for
the indicator variables and/or for the structural
equations
Analytical Procedure
• Estimate the parameters of the model with
no constraints (i.e., allow the parameters
to differ among groups)
• Compute chi-square as a measure of fit.
• Re-estimate the parameters of the model
after imposing cross-group equality
constraints on parameters
Analytical Procedure (2)
• Determine the chi square difference is
significant
• If the relative fit of the constrained model
is significantly worse than that of the
unconstrained model, then individual path
coefficients should be compared across
samples.
Structural Model Example
Lyman, DR., Moff, HT, StouthamerLoeber,M. (1993). “Explaining the Relation
Between IQ and Delinquency: Class,
Race, Test Motivation or Self-Control.”
Journal of Abnormal Psychology, 102,
187-196.
Structural Model Example:
Data
• Covariance matrices for White (n=214) and
African American (n=181) male adolescents
• Total observations: n=395
• Degrees of Freedom: 2 * 5(6) = 30
2
• 7 parameters constrained to be equal
• Variances and covariances allowed to vary
between groups.
Fit Statistics for the Multiple
Sample Model
• Χ2 = 11.68
df = 7
NS
• Χ2/df =1.67
• NFI = 0.96
• NNFI = 0.95
Modification Indices for EqualityConstrained Parameters
• MI values estimate the amount by which the
overall chi square value would decrease if the
associated parameters were estimated
separately in each group.
• Statistical significance of a modification index
indicates a group difference on that parameter
• For example, there is a statistically significant
difference on the Achievement to Delinquency
path and the Social Class to Achievement path.
Additional Analysis
• Path coefficients were estimated
separately for each sample
• Standardized values can only be used for
comparisons within a group.
• Unstandardized values are used for
comparisons between or across groups.
Results
• In both samples, Verbal Ability has a significant
effect on Achievement.
• Verbal Ability is the only significant predictor of
Delinquency in the White sample.
• Achievement is the only significant predictor of
Delinquency in the African-American sample.
• Conclusion: Among male adolescents, school
has a larger role in preventing the development
of delinquency for African-Americans that for
Whites
Use of Multiple Sample CFAs
• Test for measurement invariance, whether
a set of indicators assesses the same
latent variables in different groups.
• Examine a test for construct bias, whether
a test measures something different in one
group than in another.
Analytical Procedure
• Estimate the parameters of the model with
no constraints (i.e., allow the factor
loadings and error variances to differ
among groups).
• Compute chi square as a measure of fit.
• Re-estimate the parameters of the model
after imposing cross-group equality
constraints parameters
Analytical Procedure (2)
• Determine if the chi square difference is
significant
• If the relative fit of the constrained model
is significantly worse than that of the
unconstrained model, then individual
factor loadings should be compared
across samples to determine the extent of
partial measurement invariance..
Confirmatory Factor Analysis:
Example
Werts, CE, Rock, DA, Linn, RL and
Joreskog, KG. (1976). “A Comparison of
Correlations, Variances, Covariances and
Regression Weights With or Without
Measurement Errors.” Psychology
Bulletin, 83, 52-56.
Confirmatory Factor Analysis:
Data
• Covariance matrices are for two samples
(n1=865 and n2=900) of candidates who
took the SAT in January 1971.
• Total observations: n=1765
• Degrees of Freedom: 2 * 4(5) = 20
2
Confirmatory Factor Analysis:
Results
• The factor loadings are the same for the
two groups.
• The error variances differ between the two
groups.
Model A
• Parameters for the two groups
– Factor Loadings Equal
– Factor Correlations Equal
– Error Variances Equal
Model Fit
Chi Square = 34.89
df= 11
p < 0.00011
Model B
• Parameters for the two groups
– Factor Loadings Unequal
– Factor Correlations Equal
– Error Variances Equal
Model Fit
Chi Square = 29.67
df= 7
p < 0.00011
Model C
• Parameters for the two groups
– Factor Loadings Unequal
– Factor Correlations Equal
– Error Variances Unequal
Model Fit
Chi Square = 4.03
df= 11
p < 0.26
Chi Square difference = 29-67-4.03 =25.03
df difference = 7-3=4
Model D
• Parameters for the two groups
– Factor Loadings Equal
– Factor Correlations Equal
– Error Variances Unequal
Model Fit
Chi Square = 10.87
df= 7
p < 0.14
Chi Square difference = 34.89-10.87 =24.01
df difference = 11-7=4
James G. Anderson, Ph.D.
Purdue University
Rationale of Multiple
Sample SEMs
• Do estimates of model parameters vary
across groups?
• Another way of asking this question is:
Does group membership moderate the
relationships specified in the model?
Uses of Multiple Sample SEMs
•
•
Use for analysis of cross-sectional,
longitudinal, experimental and quasiexperimental data and to test for measurement
variance.
This procedure allows the investigator to:
1) Estimate separately the parameters for multiple
samples
2) Test whether specified parameters are equivalent
across these groups.
3) Test whether there are group mean differences for
the indicator variables and/or for the structural
equations
Analytical Procedure
• Estimate the parameters of the model with
no constraints (i.e., allow the parameters
to differ among groups)
• Compute chi-square as a measure of fit.
• Re-estimate the parameters of the model
after imposing cross-group equality
constraints on parameters
Analytical Procedure (2)
• Determine the chi square difference is
significant
• If the relative fit of the constrained model
is significantly worse than that of the
unconstrained model, then individual path
coefficients should be compared across
samples.
Structural Model Example
Lyman, DR., Moff, HT, StouthamerLoeber,M. (1993). “Explaining the Relation
Between IQ and Delinquency: Class,
Race, Test Motivation or Self-Control.”
Journal of Abnormal Psychology, 102,
187-196.
Structural Model Example:
Data
• Covariance matrices for White (n=214) and
African American (n=181) male adolescents
• Total observations: n=395
• Degrees of Freedom: 2 * 5(6) = 30
2
• 7 parameters constrained to be equal
• Variances and covariances allowed to vary
between groups.
Fit Statistics for the Multiple
Sample Model
• Χ2 = 11.68
df = 7
NS
• Χ2/df =1.67
• NFI = 0.96
• NNFI = 0.95
Modification Indices for EqualityConstrained Parameters
• MI values estimate the amount by which the
overall chi square value would decrease if the
associated parameters were estimated
separately in each group.
• Statistical significance of a modification index
indicates a group difference on that parameter
• For example, there is a statistically significant
difference on the Achievement to Delinquency
path and the Social Class to Achievement path.
Additional Analysis
• Path coefficients were estimated
separately for each sample
• Standardized values can only be used for
comparisons within a group.
• Unstandardized values are used for
comparisons between or across groups.
Results
• In both samples, Verbal Ability has a significant
effect on Achievement.
• Verbal Ability is the only significant predictor of
Delinquency in the White sample.
• Achievement is the only significant predictor of
Delinquency in the African-American sample.
• Conclusion: Among male adolescents, school
has a larger role in preventing the development
of delinquency for African-Americans that for
Whites
Use of Multiple Sample CFAs
• Test for measurement invariance, whether
a set of indicators assesses the same
latent variables in different groups.
• Examine a test for construct bias, whether
a test measures something different in one
group than in another.
Analytical Procedure
• Estimate the parameters of the model with
no constraints (i.e., allow the factor
loadings and error variances to differ
among groups).
• Compute chi square as a measure of fit.
• Re-estimate the parameters of the model
after imposing cross-group equality
constraints parameters
Analytical Procedure (2)
• Determine if the chi square difference is
significant
• If the relative fit of the constrained model
is significantly worse than that of the
unconstrained model, then individual
factor loadings should be compared
across samples to determine the extent of
partial measurement invariance..
Confirmatory Factor Analysis:
Example
Werts, CE, Rock, DA, Linn, RL and
Joreskog, KG. (1976). “A Comparison of
Correlations, Variances, Covariances and
Regression Weights With or Without
Measurement Errors.” Psychology
Bulletin, 83, 52-56.
Confirmatory Factor Analysis:
Data
• Covariance matrices are for two samples
(n1=865 and n2=900) of candidates who
took the SAT in January 1971.
• Total observations: n=1765
• Degrees of Freedom: 2 * 4(5) = 20
2
Confirmatory Factor Analysis:
Results
• The factor loadings are the same for the
two groups.
• The error variances differ between the two
groups.
Model A
• Parameters for the two groups
– Factor Loadings Equal
– Factor Correlations Equal
– Error Variances Equal
Model Fit
Chi Square = 34.89
df= 11
p < 0.00011
Model B
• Parameters for the two groups
– Factor Loadings Unequal
– Factor Correlations Equal
– Error Variances Equal
Model Fit
Chi Square = 29.67
df= 7
p < 0.00011
Model C
• Parameters for the two groups
– Factor Loadings Unequal
– Factor Correlations Equal
– Error Variances Unequal
Model Fit
Chi Square = 4.03
df= 11
p < 0.26
Chi Square difference = 29-67-4.03 =25.03
df difference = 7-3=4
Model D
• Parameters for the two groups
– Factor Loadings Equal
– Factor Correlations Equal
– Error Variances Unequal
Model Fit
Chi Square = 10.87
df= 7
p < 0.14
Chi Square difference = 34.89-10.87 =24.01
df difference = 11-7=4