digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id
Alpha coefficient as descriptive statistic analysis and paired t test as inferential statistic analysis to check the significance of the finding and the null hypothesis test
in SPSS. The study found varied result for each rater.
A. Findings 1. Descriptive Statistics
The raters were asked to grade five same essays in pre- and post- scoring with two months interval and some of them did not know that they
score the same essay. The others might know that but they totally forgot what score they gave to each essay. Here is the table of five essays in pre- and post-
scoring based on the rubric used in the assessment.
1.1.Table o f Raters’ Pre- and Post-Score of Five Same Essays
No. Essay
Content Organization
Grammar Vocabulary
Mechanic Total
PRE POST PRE POST PRE POST PRE POST PRE POST PRE POST
RATER 1
1
st
14 10
14 10
40 35
13 10
4 3
85 68
2
nd
11 11
10 10
32 30
10 11
3 3
66 65
3
rd
14 14
14 14
45 44
14 12
3 4
90 88
4
th
13 22
13 12
40 40
13 12
3 3
82 79
5
th
13 22
12 11
32 35
11 12
4 3
72 73
1
st
12.5 10.5
12 10
41 33
12.5 10
4.6 4.2
82.6 67.7
digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id
RATER 2
2
nd
10.5 10
9.5 10
35 30
11.5 10
4.3 4
70.8 64
3
rd
13 13
14 13
46 41
14 14
4.3 4
91.3 85
4
th
14 13
14 13
43 35
13 10
4.6 3.5
88.6 74.5
5
th
12 13
11 13
38 41
12 12
4.6 4
77.6 83
RATER 3
1
st
15 14
15 13
41 46
15 14
4.6 4.6
90.6 91.6
2
nd
14 12.5
14 12.5
38 40
14 14.5
3.8 4
83.8 83.5
3
rd
14 14
12.5 13
40 43
14 13
4.2 4.2
84.7 87.2
4
th
15 14
15 13
40 43
12.5 13
4.2 4.3
86.7 87.3
5
th
14 14
12 14
40 40
15 13
5 5
86 86
RATER 4
1
st
12 11.5
12 11
38 35
11 11
4 3.8
77 72.3
2
nd
11 11
11 11
35 35
10 11
4 3.7
71 71.7
3
rd
11 11
11 11
36 35
11 11
3.7 3.7
72.7 71.7
4
th
12 12
12 12
38 40
11.5 12.5
3.8 3.8
77.3 80.3
5
th
11 11
11 11
35 35
10 11
3.7 3.7
70.7 71.7
RATER 5
1
st
13 12
13 12
40 40
13 12
4 4
83 80
2
nd
10 14
13 12.5
40 36
13 12
4 4
80 78.5
3
rd
13 12.5
13 12.5
35 41
12 12.5
4 4.2
77 82.7
4
th
13 12.5
12.5 12.5
35 41
13 12.5
4.2 4
77.7 82.5
5
th
10 14
13 12.5
46 36
13 12
4 4
86 78.5
1
st
11 12.5
10.5 12
36 43
11.5 13
3.8 4.2
72.8 84.7
digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id
RATER 6
2
nd
13 11.5
12.5 11.5
41 38
12.5 10
4.3 3.8
83.3 74.8
3
rd
13 11.5
12.5 11.5
41 36
12.5 10.5
4.3 4
83.3 73.5
4
th
12.5 13
11.5 12.5
40 43
11 12.5
3.8 4.3
78.8 85.3
5
th
13 11.5
12.5 11.5
41 36
12.5 10.5
4.3 4
83.3 73.5
The table above shows the real score of English teachers’ pre- and post-
scoring. The score of each category has agreed with the rubric given. Raters had meaning that the teachers who graded the essay. It was not mentioned and
explained the teachers’ identity in detail. The important one was they had same criteria; they had gotten a degree or language certificate. In other words,
the teachers were admitted having the equal capability in English. Then, the meaning of essay number was the essay identity. Even the essays were given
the number randomly in pre- and post-scoring, the five same essays had been put specific sign to help the researcher in analyzing them. Therefore, it was
assured that those five essays in post-scoring were the same in pre-scoring. There were many marks there. The way to read the table was
horizontal, means from left to right. For example, the first column was 1
st
rater, the essay number column was 4
th
and the content column was pre = 13 and post = 22. It means that the first rater graded the content for 4
th
essay in pre-scoring was 13 but in post scoring was 22. Simply, the pre- and post-
scoring produced different result, and so on.
digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id digilib.uinsby.ac.id
By the table above, the researcher wanted to show that each rater has already changed in almost categories of the assessment. Post-score could be
higher or lower than the pre-score. There were only a few post-scores which gotten as same as the pre-score. Absolutely, it was influenced the total score
of each essay. The researcher could assure that the change was not unconsciously as some of them admitted that they forgot about the score given
indeed the essay. After collecting the data, the essay scores in the table above were
analyzed the intra-class using SPSS 23 in Cronbach alpha coefficient analysis. Cronbach alpha coefficient of the teachers’ consistency intra-rater reliability
of five papers is presented below.
4.2 Table of Cronbach Alpha Coefficient Result o f Raters’ Intra-Rater
Reliability Consistency
Content Organization Grammar Vocabulary Mechanic Total
RATER 1 0.426
0.682 0.925
0.158 - 1.333
0.832
RATER 2 0.813
0.768 0.684
0.696 - 0.468
0.722
RATER 3 0.571
-1.396 0.718
0.390 0.989
0.917
RATER 4 0.947
0.750 0.708
0.800 0.375
0.796
RATER 5 -10.435 - 0.667
- 4.850 - 2.400
- 0.667 - 3.969
RATER 6 - 3.000
- 2.444 - 3.800
- 6.087 - 10.556
-10.248
The table showed the statistical analysis of data in the previous table. The result was still formed in numerical. All five essays scores in previous
table were analyzed in SPSS 23 based on the categories. For instance, the 2
nd
