Piloting of the Rubric
the sentence
Supporting Sentences 4.
Support the topic sentence
clearly and completely
develop the topic sentence
unclear or incomplete in
developing the topic
sentence Totally not
supporting the topic sentence
5. Relevant
statement There is no
irrelevant sentence
There is one irrelevant
sentence There are two
or more irrelevant
sentences.
6. Reasoning
Thinking Persuasive,
often insightful
Adequate Inadequate,
confusing overall
7. Grammar
awareness All sentences
are complete and
grammatical, and they flow
together easily.
For the most part,
sentences are complete and
grammatical, and they flow
together easily.
Readers can follow the
writing, but they are
distracted by some
grammatical errors. Some
sentences are incomplete or
halting.
TOTAL SCORE
The result of the first part piloting process could be seen in table 4.1 below: Table 4.1 Piloting result for the first section
SCORE FOR EACH ITEM PERSON
NO. 1
2 3
4 5
6 7
TOTAL SCORE
1. 3
3 2
3 2
2 2
17 2.
3 3
2 3
2 2
2 17
3. 3
2 2
3 2
2 2
16 4.
3 2
2 3
3 2
2 17
From the result, there were two items which still had different scores. The items were the arguable statement item and relevant statement item. The four
persons who checked the reliability and validity of the rubric were also asked to give comments and suggestion by answering some questions as follows:
1. Is there any unclear description in each score of each item? If there is, please mention.
2. If there is any unclear description, please give your suggestion 3. Is there any item which should be omitted from the rubric? What is it are they?
Why? 4. Do you have any item that has not been mentioned in this rubric? What is it are
they? Why? 5. free comment:
Those questions were given to make sure which item or description that should be improved to make the rubric more reliable. The revisions could be listed as follows:
1. The word “arguable” was changed into “debatable”. It is the correct term for the argumentative paragraph.
2. There were formerly two items of grammar awareness; they were grammar awareness which measured the topic sentence’s grammar and structure and
grammar awareness which measured the supporting sentences’ grammar and structure. However, it was thought as redundant. Thus, the two items of grammar
awareness were merged into one item of mechanics and grammar which measure the whole paragraph.
3. The “coherence” item was added. An essay will not be a good essay if there is no coherence between sentences and paragraph.
4. “Mechanics” was included in the grammar item. Based on the piloting result and revision listed, the rubric was improved into:
Figure 4.2 Rubric of the argumentative paragraph
Score Descriptions No. Items
3 2
1
Score
Topic Sentence 1.
Subject and attitude
there are a subject and an
attitude There is a
subject but not the attitude
There is no topic sentence
2. Debatable
statement The topic
sentence is clear and
debatable The topic
sentence is too general and
predictable The topic
sentence is unclear or absent
Supporting Sentences 3.
Support the topic
sentence clearly and
completely develop the
topic sentence unclear or
incomplete in developing the
topic sentence Totally not
supporting the topic sentence
4. Relevant
statement There is no
irrelevant sentence
There is one irrelevant
sentence There are two or
more irrelevant sentences.
5. Reasoning
Thinking Persuasive,
often insightful Adequate
Inadequate, confusing overall
Grammar and Coherence
6. Mechanics
and Grammar
All sentences are complete
and grammatical,
and have no errors in
punctuation, capitalization,
and spelling For the most
part, sentences are complete
and grammatical,
and have maximum 4
errors in punctuation,
capitalization, and spelling
For the most part, sentences are not
complete and grammatical.
Some sentences are
incompletehaltin g and have more
than 5 errors in punctuation,
capitalization, and spelling
7. Coherence
The sentences are logically
arranged, each sentence moves
on naturally and use the
appropriate transition
signals The sentences
are logically arranged but
lack of transition
signals which make the
sentences cannot flow
together easily The reader cannot
read easily because of the
bumpy jumping ideas and
sentences. There is no transition
signal
TOTAL SCORE
The piloting result of the second part could be listed in table 4.2 below: Table 4.2 Piloting result for the second section
SCORE FOR EACH ITEM PERSON
NO. 1
2 3
4 5
6 7
TOTAL SCORE
1. 3
3 3
3 2
2 2
17 2.
3 3
3 3
2 2
2 17
3. 3
3 3
3 2
2 2
17 From the piloting result, it could be seen that the score for each item was the
same among three persons who checked the rubric. The three persons who checked the rubric in the second part were also asked to give some comments and
suggestions. However, the result was good and there was no more revision needed. The second piloting result showed that the rubric was reliable. It could be
said so because each person who checked the rubric could give the same score for each item. The rubric was also said to be valid because it had the content validity. A
rubric is said to have content validity if its content measures a representative sample of the language skill, structures, etc. with which it is meant to be concerned. “A test
is said to be valid if it measures accurately what it is intended to measure,” Hughes, 1989: 22. The rubric in this research exactly measured what was intended to be
known in this study; it was the students’ competency in writing a topic sentence and supporting sentences in an argumentative paragraph. Thus, it could be said that the
rubric which was used in this research was valid and reliable.