LOGLINEAR MODELS FOR INDEPENDENCE AND
LOGLINEAR MODELS FOR
INDEPENDENCE AND
INTERACTION IN THREE-WAY
TABLES
BY ENI SUMARMININGSIH,
SSI, MM
Table Structure For Three Dimensions
• When all variables are categorical, a
multidimensional contingency table
displays the data
• We illustrate ideas using thr threevariables case.
• Denote the variables by X, Y, and Z. We
display the distribution of X-Y cell count at
different level of Z using cross sections of
the three way contingency table (called
partial tables)
• The two way contingency table obtained
by combining the partial table is called
the X-Y marginal table (this table ignores
Z)
Death Penalty Example
Defendant’
s
race
Victim’s
Race
Death Penalty
Yes
No
Percentage
Yes
White
White
19
132
12.6
Black
0
9
0
White
11
52
17.5
Black
6
97
5.8
Black
Marginal table
Defendant’s
Race
Death Penalty
Total
Yes
No
White
19
141
160
Black
17
149
166
36
290
326
Partial and Marginal Odd Ratio
Partial Odd ratio describe the association
when the third variable is controlled
The Marginal Odd ratio describe the
association when the Third variable is
ignored (i.e when we sum the counts over
the levels of the third variable to obtain a
marginal two-way table)
Associatio
n
Variables
P-D
P-V
D-V
Marginal
1.18
2.71
25.99
Level 1
0.67
2.80
22.04
Level 2
0.79
3.29
25.90
Partial
Types of Independence
A three-way IXJXK cross-classification of response variables X, Y,
and Z
has several potential types of independence
We assume a multinomial distribution with cell probabilities {i jk},
and
The models also apply to Poisson sampling with means }.
The three variables are mutually independent when
Similarly, X could be jointly independent of Y and Z, or Z could be
jointly
independent of X and Y. Mutual independence (8.5) implies joint
independence
X and Y are conditionally independent, given Z when independence
of any one variable from the others.
holds for each partial table within which Z is fixed. That is, if
then
Homogeneous Association and
Three-Factor Interaction
Marginal vs Conditional Independence
• Partial association can be quite different
from marginal association
• For further illustration, we now see that
conditional independence of X and Y,
given Z, does not imply marginal
independence of X and Y
• The joint probability in Table 5.5 show
hypothetical relationship among three
variables for new graduate of a university
Table 5.5 Joint Probability
Major
Gender
Income
Low
High
Female
0.18
0.12
Male
0.12
0.08
Science or
Engineering
Female
0.02
0.08
Male
0.08
0.32
Total
Female
0.20
0.20
Male
0.20
0.40
Liberal Art
• association between Y=income at first
The
job(high, low) and X=gender(female, male)
at two level of Z=major discipline (liberal
art, science or engineering) is described by
the odd ratios
Income and gender are conditionally
independent, given major
Marginal Probability of Y and X
Gender
Income
low
high
Female
0.18+0.02=0.20
0.12+0.08=0.20
Male
0.12+0.08=0.20
0.08+0.32=0.40
Total
0.40
0.60
The odd ratio for the (income,
gender) from marginal table
=2
The variables are not independent
when we ignore major
• Suppose Y is jointly independent of X and
Z, so
Then
And summing both side over i we obtain
=
Therefore
So
• X and Y are also conditionally independent.
In summary, mutual indepedence of the variables
implies that Y is jointly independent of X and Z,
which itself implies that X and Y are conditionaaly
independent.
Suppose Y is jointly independent of X and Z, that
is .
Summing over k on both side, we obtain
Thus, X and Y also exhibit marginal independence
So,
• joint independence of Y from X and Z (or X
from Y and Z) implies X and Y are both
marginally and condotionally independent.
Since mutual independence of X, Y and Z implies
that Y is jointly independent of X and Z, mutual
independence also implies that X and Y are
both marginally and conditionally independent
However, when we know only that X and Y are
conditionally independent,
Summing over k on both sides, we obtain
• All three terms in the summation involve
k, and this does not simplify to marginal
independence
A model that permits all three pairs to be conditionally dependent
is
Model 8.11. is called the loglinear model of homogeneous
association or of no three-factor interaction.
Loglinear Models for Three
Dimensions
Loglinear Models
• Hierarchical
Let {ijk} denote expected frequencies.
Suppose all ijk >0 and let ijk = log ijk .
A dot in a subscript denotes the average
with respect to that index; for instance,
We set
, ,
•
The sum of parameters for any index
equals zero. That is
The general loglinear model for a three-way table is
This model has as many parameters as observations and describes
all possible positive i jk
Setting certain parameters equal to zero in 8.12. yields the models
introduced previously. Table 8.2 lists some of these models. To ease
referring
to models, Table 8.2 assigns to each model a symbol that lists the
highest-order term(s) for each variable
Interpreting Model Parameters
Interpretations of loglinear model parameters use their highestorder
terms. interpretations for model (8.11). use the two-factor
For instance,
terms to
describe conditional odds ratios
At a fixed level k of Z, the conditional association between X and Y
uses
(I- 1)(J – 1). odds ratios, such as the local odds ratios
Similarly, ( I – 1)(K – 1) odds ratios {i (j)k} describe XZ conditional
association, and (J – 1)(K – 1) odds ratios {(i) jk} describe YZ
conditional
association.
Loglinear models have characterizations using constraints on
conditional odds ratios. For instance, conditional independence of
X and Y
is
equivalent (8.11)
to {i j(k)
}=
1, i=1,
. . XZ,
. , I-1,
. . , J-1, k=1,
...,
substituting
for
model
(XY,
YZ)j=1,
into. log
i j(k) yields
K.
Any model not having the three-factor interaction term has a
homogeneous
association for each pair of variables.
For 2x2x2 tables
Alcohol, Cigarette, and Marijuana Use Example
Table 8.3 refers to a 1992 survey by the Wright State University
School of
Medicine and the United Health Services in Dayton, Ohio. The
survey asked
2276 students in their final year of high school in a nonurban area
near
Dayton, Ohio whether they had ever used alcohol, cigarettes, or
marijuana.
Denote the variables in this 222 table by A for alcohol use, C for
cigarette use, and M for marijuana use.
Table 8.5 illustrates model association patterns by presenting
estimated
conditional and marginal odds ratios
For example, the entry 1.0 for the AC conditional association for
the model (AM, CM) of AC conditional independence is the
common value of the AC fitted odds ratios at the two levels of M,
The entry 2.7 for the AC marginal association for this model is the
odds ratio
for the marginal AC fitted table
Table 8.5 shows that estimated conditional odds ratios equal 1.0
for each
pairwise term not appearing in a model, such as the AC
association in model
For
that
model, the estimated marginal AC odds ratio differs from
( AM,
CM).
1.0, since conditional independence does not imply marginal
independence.
Model (AC, AM, CM) permits all pairwise associations but
maintains
homogeneous
odds ratios odds
between
two
at each level of
The AC fitted conditional
ratios
forvariables
this model
the
third.
equal
7.8.calculate this odds ratio using the model’s fitted values
One can
at either level of M, or from (8.14) using
INFERENCE FOR LOGLINEAR
MODELS
Chi-Squared Goodness-of-Fit Tests
As usual, X 2 and G2 test whether a model holds by comparing cell
fitted
values to observed counts
=2
Where
nijk = observed frequency and =expected frequency
Here df equals the number of cell counts minus the number of
model parameters.
For the student survey (Table 8.3), Table 8.6 shows results of
testing fit for
several loglinear models.
Models that lack any association term fit poorly
The model ( AC, AM, CM) that has all pairwise associations fits
well (P=
0.54)
It is suggested by other criteria also, such as minimizing
AIC= - 2(maximized log likelihood - number of parameters in
model)
or equivalently, minimizing [G2- 2(df)].
INDEPENDENCE AND
INTERACTION IN THREE-WAY
TABLES
BY ENI SUMARMININGSIH,
SSI, MM
Table Structure For Three Dimensions
• When all variables are categorical, a
multidimensional contingency table
displays the data
• We illustrate ideas using thr threevariables case.
• Denote the variables by X, Y, and Z. We
display the distribution of X-Y cell count at
different level of Z using cross sections of
the three way contingency table (called
partial tables)
• The two way contingency table obtained
by combining the partial table is called
the X-Y marginal table (this table ignores
Z)
Death Penalty Example
Defendant’
s
race
Victim’s
Race
Death Penalty
Yes
No
Percentage
Yes
White
White
19
132
12.6
Black
0
9
0
White
11
52
17.5
Black
6
97
5.8
Black
Marginal table
Defendant’s
Race
Death Penalty
Total
Yes
No
White
19
141
160
Black
17
149
166
36
290
326
Partial and Marginal Odd Ratio
Partial Odd ratio describe the association
when the third variable is controlled
The Marginal Odd ratio describe the
association when the Third variable is
ignored (i.e when we sum the counts over
the levels of the third variable to obtain a
marginal two-way table)
Associatio
n
Variables
P-D
P-V
D-V
Marginal
1.18
2.71
25.99
Level 1
0.67
2.80
22.04
Level 2
0.79
3.29
25.90
Partial
Types of Independence
A three-way IXJXK cross-classification of response variables X, Y,
and Z
has several potential types of independence
We assume a multinomial distribution with cell probabilities {i jk},
and
The models also apply to Poisson sampling with means }.
The three variables are mutually independent when
Similarly, X could be jointly independent of Y and Z, or Z could be
jointly
independent of X and Y. Mutual independence (8.5) implies joint
independence
X and Y are conditionally independent, given Z when independence
of any one variable from the others.
holds for each partial table within which Z is fixed. That is, if
then
Homogeneous Association and
Three-Factor Interaction
Marginal vs Conditional Independence
• Partial association can be quite different
from marginal association
• For further illustration, we now see that
conditional independence of X and Y,
given Z, does not imply marginal
independence of X and Y
• The joint probability in Table 5.5 show
hypothetical relationship among three
variables for new graduate of a university
Table 5.5 Joint Probability
Major
Gender
Income
Low
High
Female
0.18
0.12
Male
0.12
0.08
Science or
Engineering
Female
0.02
0.08
Male
0.08
0.32
Total
Female
0.20
0.20
Male
0.20
0.40
Liberal Art
• association between Y=income at first
The
job(high, low) and X=gender(female, male)
at two level of Z=major discipline (liberal
art, science or engineering) is described by
the odd ratios
Income and gender are conditionally
independent, given major
Marginal Probability of Y and X
Gender
Income
low
high
Female
0.18+0.02=0.20
0.12+0.08=0.20
Male
0.12+0.08=0.20
0.08+0.32=0.40
Total
0.40
0.60
The odd ratio for the (income,
gender) from marginal table
=2
The variables are not independent
when we ignore major
• Suppose Y is jointly independent of X and
Z, so
Then
And summing both side over i we obtain
=
Therefore
So
• X and Y are also conditionally independent.
In summary, mutual indepedence of the variables
implies that Y is jointly independent of X and Z,
which itself implies that X and Y are conditionaaly
independent.
Suppose Y is jointly independent of X and Z, that
is .
Summing over k on both side, we obtain
Thus, X and Y also exhibit marginal independence
So,
• joint independence of Y from X and Z (or X
from Y and Z) implies X and Y are both
marginally and condotionally independent.
Since mutual independence of X, Y and Z implies
that Y is jointly independent of X and Z, mutual
independence also implies that X and Y are
both marginally and conditionally independent
However, when we know only that X and Y are
conditionally independent,
Summing over k on both sides, we obtain
• All three terms in the summation involve
k, and this does not simplify to marginal
independence
A model that permits all three pairs to be conditionally dependent
is
Model 8.11. is called the loglinear model of homogeneous
association or of no three-factor interaction.
Loglinear Models for Three
Dimensions
Loglinear Models
• Hierarchical
Let {ijk} denote expected frequencies.
Suppose all ijk >0 and let ijk = log ijk .
A dot in a subscript denotes the average
with respect to that index; for instance,
We set
, ,
•
The sum of parameters for any index
equals zero. That is
The general loglinear model for a three-way table is
This model has as many parameters as observations and describes
all possible positive i jk
Setting certain parameters equal to zero in 8.12. yields the models
introduced previously. Table 8.2 lists some of these models. To ease
referring
to models, Table 8.2 assigns to each model a symbol that lists the
highest-order term(s) for each variable
Interpreting Model Parameters
Interpretations of loglinear model parameters use their highestorder
terms. interpretations for model (8.11). use the two-factor
For instance,
terms to
describe conditional odds ratios
At a fixed level k of Z, the conditional association between X and Y
uses
(I- 1)(J – 1). odds ratios, such as the local odds ratios
Similarly, ( I – 1)(K – 1) odds ratios {i (j)k} describe XZ conditional
association, and (J – 1)(K – 1) odds ratios {(i) jk} describe YZ
conditional
association.
Loglinear models have characterizations using constraints on
conditional odds ratios. For instance, conditional independence of
X and Y
is
equivalent (8.11)
to {i j(k)
}=
1, i=1,
. . XZ,
. , I-1,
. . , J-1, k=1,
...,
substituting
for
model
(XY,
YZ)j=1,
into. log
i j(k) yields
K.
Any model not having the three-factor interaction term has a
homogeneous
association for each pair of variables.
For 2x2x2 tables
Alcohol, Cigarette, and Marijuana Use Example
Table 8.3 refers to a 1992 survey by the Wright State University
School of
Medicine and the United Health Services in Dayton, Ohio. The
survey asked
2276 students in their final year of high school in a nonurban area
near
Dayton, Ohio whether they had ever used alcohol, cigarettes, or
marijuana.
Denote the variables in this 222 table by A for alcohol use, C for
cigarette use, and M for marijuana use.
Table 8.5 illustrates model association patterns by presenting
estimated
conditional and marginal odds ratios
For example, the entry 1.0 for the AC conditional association for
the model (AM, CM) of AC conditional independence is the
common value of the AC fitted odds ratios at the two levels of M,
The entry 2.7 for the AC marginal association for this model is the
odds ratio
for the marginal AC fitted table
Table 8.5 shows that estimated conditional odds ratios equal 1.0
for each
pairwise term not appearing in a model, such as the AC
association in model
For
that
model, the estimated marginal AC odds ratio differs from
( AM,
CM).
1.0, since conditional independence does not imply marginal
independence.
Model (AC, AM, CM) permits all pairwise associations but
maintains
homogeneous
odds ratios odds
between
two
at each level of
The AC fitted conditional
ratios
forvariables
this model
the
third.
equal
7.8.calculate this odds ratio using the model’s fitted values
One can
at either level of M, or from (8.14) using
INFERENCE FOR LOGLINEAR
MODELS
Chi-Squared Goodness-of-Fit Tests
As usual, X 2 and G2 test whether a model holds by comparing cell
fitted
values to observed counts
=2
Where
nijk = observed frequency and =expected frequency
Here df equals the number of cell counts minus the number of
model parameters.
For the student survey (Table 8.3), Table 8.6 shows results of
testing fit for
several loglinear models.
Models that lack any association term fit poorly
The model ( AC, AM, CM) that has all pairwise associations fits
well (P=
0.54)
It is suggested by other criteria also, such as minimizing
AIC= - 2(maximized log likelihood - number of parameters in
model)
or equivalently, minimizing [G2- 2(df)].