Instrument of Data DATA COLLECTION
56
Practically, DCT can be either written tasks or presented orally. In gathering the data using DCT, it constructs a type of language questionnaire whereby it is able
to elicit what the speakers thought they would say in such situation. Basically, the way to gain the data is through setting a scenario which is presented and the
informants or the respondents have a choice to give their responses spontaneously by referring to the speech acts they opt for, that are for requests. As the attempt to
enhance the result of the DCT, Yoon and Kellogg 2002 propose to add a feature by using a cartoon picture to provide pictorial context allowing the respondents to
elaborate the speech acts naturally. Before tested to the
research’s subjects, the DCTs as the research instrument had to meet the validity and reliability of the instruments. Kraska-Miller 2014:18
unde rlines that “validity and reliability are the key psychometric properties related
to measurement instruments”. At this point, validity extends to which an instrument used to measure what it is supposed to measure. Therefore, in this study, the
researcher measured the validity and reliability of the DCT as the data elicitation instrument.
It has been mentioned above that basically validity referred to the degree to which an instrument appears to measure what it purports to measure. Thus, the
researcher first assessed the validity in terms of construct validity which covers also the face and content validity as well as instrument reliability. Kraska-Miller
2014:19 defines that “face validity is a qualitative decision related to the appearance, relevance, and representativeness of the
items on a written instrument”. By taking four member panel as judges or raters, the researcher assured that the
57
DCTs are indeed professional, user friendly, understandable, and easy to administer. In the following step of validity, content validity took a place. Content
validity made possible for the researcher to find out the entire content of the behavior or the area that is represented in the test. According to Kraska-Miller
2014:20, content validity concerns the completeness, correctness, and clarity of the items concerning that the content of instrument is ethical, professional, relevant,
and representative to be tested. Since this research focused on finding whether different fields of study contribute to the selection of the speech acts, thus, in the
DCTs the researcher set ten real working life situations as the contexts. These all specified situations were adopted and modified from the previous request acts
research conducted by Rue Zhang 2008, and as to meet the content validity, the researcher gave each situation a cartoon picture from the internet that matched the
contexts. As the raters, the researcher took four students from English Language Studies ELS of Sanata Dharma University whom considered as the experts in
making English request since they can be categorized as advance learners of English. As the response, the ELS students gave valuable feedback on the DCTs
used in this study that helped the researcher to make corections, changes, as well as suggestions to develop the DCTs as the instrument to gather the data.
Furthermore, afterall, the focus of the validity of the research instrument was the construct validity. According to Kraska-Miller 2014:20 construct validity
is more elusive than the face and content validity and often it is difficult to define construct. In the same way, Pallant 2011:7 mentions that construct validity
requires a test to make it valid concerning the nature of the underlying construct. In PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
58
addition, Zumbo and Chan 2014:20 accentuate construct validity as the extent to which relation among items is usually established through a series of statistical
using correlation procedure. Equally, besides validity, the instrument of the research has also to meet the reliability matter.
Kraska- Miller 2014:20 contrues that reliability deals with “degree to
which a measuring instrument is consistent over time on measures for similar populations”. As to encounter the reliability of the DCTs employed in this research,
the researcher measured them in the standard of the inter-rater reliability. Kraska- Miller 2014:21 implies that inter-rater reliability is the degree to which the ratings
of two or more judges or raters are similar for the same object or performance. In meeting the construct validity and reliability of the DCTs, one of the most
commonly used indicators is the Cronbach’s alpha coefficient and ideally the coefficient of scale should be above 0.70 Pallant, 2011:97. To emphasize, Perron
and Gillespie 2015:146 mention that “high level of internal consistency Cronbach’s alpha 0.90 provides strong evidence of reliability and validity”.
Thefore, from the four raters, the researcher tested the result of the inter-rater reliability in form of ten items or situations of DCTs on real working life setting
with its response. The result is displayed in the following table.
Table 6. Result of DCT’s validity and reliability
59
The table above displayed the Cronbach’s alpha α = 0.713 which indicated that it fulfilled the minimum coeffiecient to be reliable and valid. Thus, the DCT
instrument allowed to be used for this study. On the other hand, referring to the comments and suggestions form the four raters which resulted various rating for the
first situation which considered not really fulfilled the contexts of the real life working setting. Thus, as focal points to refer, the researcher did corrections and
improvements on the DCTs used in this research. In turns, the researcher refined it and finally selected nine situational contexts as the realization of requests for the
students of Medicine and Law to be fulfilled. The researcher made sure that the items were clear, complete and correct, so did the most corresponding cartoon
picture as the pictorial situations that supported and stimulated the students in figuring out the contexts situated for the DCTs, thus, they were able to give
responses appropriately toward the state occured. The nine situated contexts portrayed simple every day office life and working conditions which commonly
happen. One of the example is displayed in figure 3 below.
Figure 3. Sample of DCT illustration with a cartoon picture Adapted and developed from Rue and Zhang 2008
60
In short, the steps on validity and reliability are summarized into three stages, namely: 1 Designing DCTs. In this stage, the researcher designed 10
situations dealt with working life and office things in form of clear and correct illustration or decription for each situational context and provided a cartoon picture
as pictorial context to ease the students in imagining the conditions they were to response. 2 Testing the DCTs. At this stage, to check whether the DCTs work or
can be used as the instrument to gather the data and help the respondents to response appropriately, the researcher tested the validity and reliability of them by
distributing the DCTs to English students as the raters and judges. Here, the researcher summarized the result into table and inputted them into SPSS by using
scales. Then, the following test conducted was the Cronbach’s alpha test and it
resulted α = 0.713 which confirmed that the DCT was valid and reliable. And, 3
Refining DCTs. Developed from Rue and Zhang 2008 form of DCTs and had been tested to raters, DCTs were given some valuable and notable feedbacks,
corections, and suggestions by the raters, then, the researcher refined them concerning the correctness, completeness, and clarity of each situations and decided
to take nine situational settings to be distributed to Medicine and Law students. The details of each situation are explained in table 7.
61
Table 7. Discourse Completion Task Situational Contexts Situation
Description and illustration Context
1 You have something urgent to do while you have a
morning shift. You want to ask your colleague to substitute his shift with you. How do you ask for his
help? -Distance
-Power -Rank
persuade, informal
2 You are on your way out of a company building and both
your hands are so full that you need someone to open the door for you. You notice that there is an elderly person
near the door. What would you say to get this person to open the door for you?
+Distance -Power
+Rank persuade, informal
3 You work as the team leader, during a meeting with all
the staff, you are very thirsty and there is a new staff sitting next to you. What would you say to get this staff
to bring you a glass of water? +Distance
+Power +Rank
order, formal
4 Your colleague is borrowing your computer since his is
broken. But, at the same time, you need to finish your tasks because the boss is waiting for you. How would
you ask him to return your computer? -Distance
-Power -Rank
persuade, informal
5 You are talking with your boss. You would like to
change job, because you feel that you cannot cope with the responsibilities on your present position. What would
you ask your boss about it? +Distance
-Power +Rank
persuade, formal
6 You are waiting for a job interview in a crowded non-
smoking office. Another person enters the office and after a while he lights a cigarette. What would you say to
get this person to put his cigarette out? +Distance
-Power -Rank
warn, informal
7 You need to photocopy a lot of documents for a
meeting, and only have 15 minutes before the meeting starts. However, your colleague is using the
photocopier and he has many documents to finish too. How would you ask him whether or not you can interrupt
and do your photocopying first? -Distance
-Power -Rank
+Force urgent, informal
8 You have a colleague whom you are not so close to. He
has just got a promotion. How would you ask him to treat you and other office members to a meal?
+Distance -Power
-Rank persuade, informal
9 You are a manager and you know that your staff has done
a great job in managing marketing. You have a big project coming up which needs this capable person to
manage it. What would you ask him to handle this project?
+Distance +Power
+Rank persuade, formal
62