Development of the Message Evaluation Instruments

27 Table 3.3 Nation Vocabulary Test’s 2007 Word Target Level Word Target 1 st 1,000 see, time, period, figure, poor, drive, jump, shoe, standard, basis. 2 nd 1,000 maintain, stone, upset, drawer, patience, nil, pub, circle, microphone, pro. 3 rd 1,000 soldier, restore, jug, scrub, dinosaur, strap, pave, dash, rove, lonesome. 4 th 1,000 compound, latter, candid, tummy, quiz, input, crab, vocabulary, remedy, allege. 5 th 1,000 deficit, weep, nun, haunt, compost, cube, miniature, peel, fracture, bacterium 6 th 1,000 devious, premier, butler, accessory, threshold, thesis, strangle, cavalier, malign, veer. 7 th 1,000 olive, quilt, stealth, shudder, bristle, bloc, demography, gimmick, azalea, yoghurt. 8 th 1,000 erratic, palette, null, kindergarten, eclipse, marrow, locust, authentic, cabaret, mumble. 9 th 1,000 hallmark, puritan, monologue, weir, whim, perturb, regent, octopus, fen, lintel. 10 th 1,000 awe, peasantry, egalitarian, mystique, upbeat, cranny, pigtail, crowbar, ruck, lectern. 11 th 1,000 excrete, mussel, yoga, counterclaim, puma, pallor, aperitif, hutch, emir, hessian. 12 th 1,000 haze, spleen, soliloquy, reptile, alum, refectory, caffeine, impale, coven, trill. 13 th 1,000 ubiquitous, talon, rouble, jovial, communique, plankton, skylark, beagle, atoll, didactic. 14 th 1,000 canonical, atop, marsupial, augur, bawdy, gauche, thesaurus, erythrocyte, cordillera, limpid.

b. Questionnaire

The second test was the expert validation which involved three ELESP Sanata Dharma lecturers who teach vocabulary. The test instruments used by the researcher was in a form of a questionnaire. The lecturers were asked to evaluate and give feedback to the designed test. The researcher used close-ended questions 28 in a form of Likert Scale and open-ended questions. A Likert Scale was used in the close-ended questions. The expert validation instrument contained 12 close-ended questions and 2 open ended questions. The 2 open ended questions were provided in order to get deeper evaluation from the expert.

E. Data Analysis Technique

There were two ways of analyzing the data. For the Vocabulary Size Test, the researcher summed up the correct answers from the test and multiplied them by 100 as stated in Nation 2012. The result of the test pictured the current vocabulary size of the sixth semester students. Then, the researcher calculated the score of the vocabulary size test by using a computer program namely Range for texts v.3 which is found in Compleat Lexical Tutor website www.lextutor.ca. The researcher put the students’ answer in this program. Then, this program will automatically give the total of the correct answer of each levels. To measure the vocabulary size of the sixth semester students, the researcher multiplied the total correct answer by 100 as stated in Nation 2012. The researcher then tried to find out the mean of the preliminary test result in order to find out the data distribution of the test by using the Microsoft Excel. The researcher compiles the vocabulary size of the sixth semester students and find the mean by applying the mean formula. Next, the questionnaire for expert validation was analyzed by calculating the total scores from the Likert Scale. The scores then were drawn in the percentage and description of words. The results were the validity of test from the experts. 29

F. Research Procedures

The last part of the research showed the procedures of designing the test material. There are only 3 steps from Morrison, Ross, Kalman, and Kemp’s 2011 model used by the researcher in designing the test material since there are no specific steps to follow. Therefore, the researcher is very flexible in deciding the steps. Each step is explained as follows:

1. Learners and Contexts

In this step, the researcher decided the learners’ characteristics which were going to be discussed. The preliminary test was used by the researcher to determine the vocabulary size of the sixth semester students. Then, the researcher was able to design the test after gathering the result of the preliminary test.

2. Development of the Instrument

There were 4 steps in developing the insturment. First, the designer took samples of the test items from the word lists. The test items were chosen by interval sampling from the word lists. Secondly, the researcher made the stem. It was in the form of target words that the test-takers have to answer and the format of the test. Third, the researcher made the distractors. As suggested by Nation 2012, they were taken from the same level of words. Lastly, the orderlevel of test items had to take into account by the researcher.

3. Evaluating the Instrument

After designing the test materials, the researcher distributed the questionnaires to the three experts of vocabulary. They were the lecturers of ELESP