128
No. 38 Successful
The instruction should be clearer No. 39
Successful The instruction should be clearer
It is shown in table that there is no conformity between those two aspects. While all items are correctly responded by the students, on the contrary they are
suggested for revision by the experts. Therefore, the decision of revision will be made by also digging out the improvement feedbacks given by the experts that
will be elaborated in the discussion of the preliminary product revision.
3. Overview of the Design of the Testing Model.
To have more comprehensive evaluation on the design of the testing model, the researcher invites the expert and student respondents’ opinion on the design of
the testing model and observes the time length needed by the students to complete the tasks during the trial test to formulate time duration of the test model. They
will also be considered to the improvement of the preliminary product beside the evaluation of the formulation of the indicators of language competence and the
congruence of indicators of language competence with test items and the overview design of the testing model. Therefore the discussion of the overview of the
design will be divided into three parts:
a. Overview of Time Duration
The formulation of time duration of the test model is made based on the researcher’s observation on the time consumed by the test takers to complete the
test tasks during the trial test. The preliminary test prototype was designed for being completed in 60 minutes. Yet, to formulate the right duration, the students
129 were instructed to start working on the test at the same time and reported to the
researcher the time they finished completing the tasks. Here is the recap:
No Name of
students Level of
English competence
Duration of test completion
1. Aisyah
High 10.55 up to 11.15
20 minutes
2. Zahid Yahya
High 10.55 up to 11.16
21 minutes
3. Sukhan Ainur
High 10.55 up to 11.15
20 minutes
4. Atifah Fauzan
High 10.55 up to 11.16
21 minutes
5. M.Yasin
Middle 10.55 up to 11.23
28 minutes
6. Laily Maghfira Middle
10.55 up to 11.20
25 minutes
7. Ahmad Faqih
Middle 10.55 up to 11.21
26 minutes
8. Nur Annisa
Middle 10.55 up to 11.25
30 minutes
9. Naufal D.R.
Low 10.55 up to 11.15
21 minutes
10. Nida Fairus K Low
10.55 up to 11.25
30 minutes
11. Bligania Low
10.55 up to 11.27
32 minutes
12. Fikri N. Low
10.55 up to 11.20
25 minutes Average time consumed is 24.9 minutes
The above report shows that the average time the students need to complete the tasks is about 24. 9 minutes. This data will be analyzed together with the
researcher’s estimation, and results of the questionnaire filled by the expert and student respondents to formulate the right duration of completing the test tasks as
the improvement feedback of the testing model.
Table. 4.15. Time consumption of completing the test tasks
130
b. Overview of the Design from the Experts’ Point of View
Ten questions are given to the experts to dig out their opinions on the design of the test model. The questions are formatted as open questions that require the
experts to explain their opinion as much as possible. The answers are considered as feedbacks to improve the test model and are presented below:
Question 1: Is the test model appropriate for the age and grade
level of the test takers?
Expert 1 Expert 2
Expert 3 Expert 4
Yes and No, There should
be age
requirement to take the test because
what an 8 year-old can do is greatly
different from what a-9-year old can do
No comment yes
Yes, For
elementary school students,
the test model is appropriate. The
materials covered are topics closely
related to the test takers
Concluded judgment :
The test model is quite appropriate for the age and grade level of the test takers.
Question 2: Is the test model appropriate for use with the English
educational backgroundsituation of the test takers?
Expert 1 Expert 2
Expert 3 Expert 4
No, academically
speaking, the test does not show an
appropriate use of English
Yes, but types of genre needs to
be enriched such as; TV menu,
short assignment,
signs, because English for the
test
takers is used
as ‘survival’ tool
yes Yes
Concluded judgment :
The test model is actually quite appropriate for use with the English educational background situation of the test
takers, yet the topics or genres should be more varied.
Table. 4.16. Improvement feedbacks from expert respondents
131
Question 3: Is the general level of proficiency required by the test
suitable for the test takers?
Expert 1 Expert 2
Expert 3 Expert 4
Yes and No, it depends on the age
of the test takers. Children are still
growing; they vary in term of cognitive
development, social and psychological
development. Yes
Yes Yes
Concluded judgment :
The general level of proficiency required by the test is quite suitable for the test takers.
Question 4: Is the test model acceptable interestingattractive to the
test takers ?
Expert 1 Expert 2
Expert 3 Expert 4
Yes. Interesting and attractive.
The illustration helps a
lot. Yet, the tasks can be more ‘hands
on’ so that they can be more engaging
for children Yes
Yes, the colorful pictures make the
test booklet and the answer sheet
attractive
and interesting. Yes,
the layouts of the test booklet and
the answer sheet are
similar. Should find a
way to modify the layout so that
there aren’t many repeated
arrangements to create the more
efficient answer sheettest
booklet Yes,
provided with
simple sentences, colored
pictures, this test model
is interesting enough
for the test takers
Concluded judgment :
The test model is acceptable interestingattractive to the test takers.
Question 5: Is there sufficient time to give the test?
Expert 1 Expert 2
Expert 3 Expert 4
No, perhaps, 35’ or 40’ not 30’
No, 30 minutes for 39 questions
are too short. No,
Suggested time
allotment: 45-50
minutes. Yes, I think 30
minutes is enough to take the test.
132 Some questions
need longer time to do especially
re-writing considered
enough for the test takers to
readunderstand the instructions
and to do the tests
The task types given matching
words, true and false,
multiple choices,
gap filling and text
writing allow the test takers to do it
with
the given time.
Concluded judgment :
Time to give the test is not sufficient.40-50 would be enough
Question 6: Are all materials and equipment needed to give the test
available?
Expert 1 Expert 2
Expert 3 Expert 4
Yes, the
test provides all the
necessary items for the
test administration.
Yes, assuming that the students
provide their
own pen
or pencil.
Yes, All
materials and
equipment are
available. The
scoring system
should be put in the answer key
only, no need to put it in the test
booklet
and answer sheet.
Yes, the model has been
provided with the test sheet,
the answer sheet and the guidelines
for evaluating the testing model
Concluded judgment :
The materials and equipments needed to give the test are available.
Question 7: Is the purpose of the test clear and appropriate?
Expert 1 Expert 2
Expert 3 Expert 4
I am skeptical with the term of ‘making
decision’ purpose. What
kind of
decision? How will the decision affect
childrenchildren’s future?
Yes and No, some of the test
questions do not reflect
the purpose
Yes Yes,
Being designed to be a
reference in
evaluating English teaching program
in the level of elementary school,
the model test has met the clear and
appropriate purpose.
Concluded judgment :
The purpose of the test is not quite clear and appropriate; it needs a better formulation of the purpose on the test
making.
133
Question 8: Are the test instructions complete, clear and specific?
Expert 1 Expert 2
Expert 3 Expert 4
Yes. Yet, you need to give examples of
how to do the test items
in every
section Yes
Yes, the
instructions are specific and easy
to understand. Yes, in general,
the test
instructions are complete, clear
and specific. Part 3
of reading
comprehension and part 1 of
writing are an exception. They
put
wrong numbers as a part
of the
instructions.
Concluded judgment :
The test instructions actually are complete, clear and specific, yet the format of the test model should be made
more carefully in the detail.
Question 9: Do you think it is necessary to give translations on the
English instructions in bahasa Indonesia?
Expert 1 Expert 2
Expert 3 Expert 4
No, translations are needed as long as
there is an example of how to do the test
items
in every
section No.
No, not only that the instructions
are already
specific and easy to understand, but
also that
the examples given
help the
test takers in doing
the test No,
the instructions are
so clear that the test takers do not
need
the translation. The
example given in each part will
make the test takes
more understand what
to do
Concluded judgment :
No translation in bahasa Indonesia for the instructions is needed
Question 10: Is the layout clear and interesting for the test takers?
Expert 1 Expert 2
Expert 3 Expert 4
Yes, Kids would love the illustration
and the lay out Yes and No,
some pictures
are not clear and the font is too
small Yes, overall, the
layout is
interesting. However, some
parts need to modify
eg. No, I don’t quite
agree with the layout. It’s too
small to see and read. The choice
of font size needs
134 Answer
sheet writing part 2:
lines should be provided as the
space to write the answers.
Another suggested layout:
see the notes in ‘answer
sheet’ and
the test
booklet to
be differentiated
between the
instructions and the questions.
pictures and
instruction larger
Concluded judgment :
The layout of the testing instrument is quite clear and interesting for the test takers, but some formats need a
little revision.
Question 11: Are the test items unbiased and culturally appropriate?
Expert 1 Expert 2
Expert 3 Expert 4
Yes, both genders are
represented equally. What if the
characters represent more of Indonesian
culture? What
if the
characters represent accommodate
the disabled?
No comment Yes,
culturally appropriate
although a
‘school bus’
might be
something ‘rare’ in Indonesia.
Yes, in
my opinion, the test
items are
unbiased and
culturally appropriate with
the choice of characters’
names in the text and
questions. The context is
clear and familiar to the test takers.
Concluded judgment :
The test items are culturally unbiased and appropriate
Question 12: Do you think it is necessary to provide a technical test
manual?
Expert 1 Expert 2
Expert 3 Expert 4
Yes, if it is meant for mass use. There
should be a manual that tells the test
users
about the
following info: - the purpose
- the test takers - what does the test
Yes, I think you can allow them
to practice first with a clear
instruction and explanation
Yes, It
is important
for those
who administer
the test; to help them
administer the
test properly and give information
to the test takers Yes.
135 results
- how to use the test results.
correctly.
Concluded judgment :
A technical test manual is necessary to provide.
The data above is analyzed and concluded as the overview feedbacks to improve the design of the testing model. The concluded feedbacks are: 1 the test
model is quite appropriate for the age and grade level of the test takers. 2 The test model is actually quite appropriate for use with the English educational
background situation of the test takers, but the topics or genres should be more varied. 3 The general level of proficiency required by the test is quite suitable
for the test takers. 4 The test model is acceptable interestingattractive to the test takers. 5 Time to give the test is not sufficient; 40-50 would be enough. 6
The materials and equipments needed to give the test are available. 7 The purpose of the test is not quite clear and appropriate; it needs a better formulation
of the purpose on the test making. 8 The test instructions actually are complete, clear and specific, but the format of the test model should be made more carefully
in the details. 9 No translation in bahasa Indonesia for the instructions is needed. 10 The layout of the testing instrument is quite clear and interesting for the test
takers, but some formats need a little revision. 11 The test items are culturally unbiased and appropriate. 12 A technical test manual is necessary to provide.
c. Overview of the Design from the Test Taker Candidates Students
