AN INVESTIGATION OF THE CONSTRUCTION OF THE ENGLISH FINAL TESTS AND THE TESTS’ COMPLIANCE WITH THE OBJECTIVES OF THE PROGRAM : A Case Study at Three Primary Schools in Purworejo.

TABLE OF CONTENTS

Approval Sheet……….i

Declaration………. ii

Acknowledgements………..…...iii

Abstract ………v

Table of Contents ………...vi

Lists of Tables ………..…………..………ix

Chapter I INTRODUCTION……….……..………..1

I.1 Background of the Study ... 1

I.2 Purposes of the Study ... 4

I.3 Research Questions ... 5

I.4 Significance of the Study ... 5

I.5 Scope of the Study ... 7

I.6 Definition of the Terms……….…….8

I.7 Organization of the Thesis……….…...10

Chapter II REVIEW OF RELATED LITERATURE ……….…...….12

2.1 Language Testing ... .12

2.2 Qualities of Language Testing ... .16

2.3 The Validity of Language test……. ... 19

2.4 Achievement Test on English as a foreign language …………..…….…..22

2.4.1 Testing Vocabulary………...24

2.4.2 Testing Grammatical Structure……….25

2.5 The Construction of Language test………..….. 26

2.5.1 Planning the Test ………..….. 27

2.5.3 Submitting the Test Material to Review and revising on the

Basis of Review ………..….… 29

2.5.4 Pretesting the Materials and Analyzing the Result ……….……. . 30

2.5.5 Assembling the Final Form of the Test ……….... . 31

2.5.6 Producing the Test ……….…….. .. 32

2.6 The English Program……….….……... 33

2.7 Review of the Related Studies on Language Testing…………..……... 35

Chapter III RESEARCH METHODOLOGY ……….…...36

3.1 Research Design ... 36

3.2 Setting ... 39

3.3 Participants ... 40

3.4 Data Collection Techniques ... 41

3.5 Data Analysis ... 43

Chapter IV RESEARCH FINDINGS AND DISCUSSIONS ...……….46

4.1 The Findings and the Discussion of the Data from the Interview……... 46

4.1. 1 The Findings on How the English Final test were Constructed by the Teachers……….…….. . 52

4.1.2 The Discussion on How the English Final Tests were Constructed by the Teach……….……….... .. 58

4.2 The Findings and the Discussion of the data from the Documents ……... 64

4.2.1 The Objectives of the English Program conducted by the Schools ……….………..………...….…. .. 65

4.2.2 The Tests’ Items Constructed by the Teachers………...… .. 68

4.2.3 The Discussion on the Test’s Correspondence to The Objectives of the Program ………... ... 69 4.2.3.1 The Discussion of the Correspondence between the English

Final Test constructed by R1 and the Objectives of

the English Program ………. ... 70

4.2.3.2 The Discussion of the Correspondence between the English Final Test constructed by R2 and the Objectives of the English Program ………. ... 80

4.2.3.3 The Discussion of the Correspondence between the English Final Test constructed by R3 and the Objectives of the English Program ………. ... 91

Chapter V CONCLUSIONS AND SUGGESTIONS ………..108

5.1. Conclusions ………..…..…108

5.2. Suggestions ………...……...110

REFERENCES……….... .. 111

LIST OF TABLES

Table 1. Steps of activities in constructing the English final tests

conducted by the respondents ………...59 Table 2. Stages of constructing English test (Harris, 1969:95) ……….60 Table 3. The Relevance between the stages in constructing the

English Test and the Stages of test Constructions

formulated by Harris (1969:95) ...61 Table 4. The correspondence between the tests (1) and objectives

of the English program………...102 Table 5. The correspondence between the tests (1) and objectives

of the English program………...103 Table 6. The correspondence between the tests (1) and objectives

1

CHAPTER I

INTRODUCTION

This chapter focuses on the background of the study which presents ideas reasoning the study, continued by presenting the research questions and the purposes of the study. While the significance of the study, the scope of the study, and the definitions of the terms are described later. To give such a brief description of the structure of this research report, then, the organization of the thesis is also illustrated briefly in the end of this chapter.

1.1. Background of the Study

English teaching and learning practice has been developing through times. The development is indicated by many innovations and improvements (Iwamura, 2006). The improvements in English language education seem vary, ranges from the most philosophical aspects such as the view of English as the target language to the more specific aspects of teaching practices, such as the teaching approaches and methodologies and so on (Rausch, 2000). In the particular context of English as a foreign language education in Indonesia, it is indicated by improvement in English education which always takes places from time to time (Wijanarko, 2007).

The improvements evidentially indicate that the government gives attention to English teaching as one of language teaching practices and that the

2

need of the proficiency of the target language is progressively getting demanding for the community (Nasution, et.al, 2009). In line with this idea, Alwasilah (2000:27) says that language education is aimed at developing people’s language skills for various purposes.

Rausch (2000) says that language testing is one of the aspects in the English language education which is also improved in harmony with the development in English teaching. English language testing’s fundamental roles have been shown by the great attention of teachers, learners, government, and society especially to some definite form of tests, for example on some high-stakes tests such as final tests at schools, university entrance tests, job entrance tests, and so on. These facts are relevant with Harris’s (1969) statement that English test is an undeniable need in educational practice which provides some principal uses in educational programs. Since test seems as unavoidable as tomorrow’s sunrise in virtually every kind of educational setting (Brown, 2004:1), it is impossible to position test far from teaching and learning practice. They are closely related and the relationship has some principal functions. In line with these ideas, McNamara (2000) also gives some arguments. First, language test plays a powerful role in many people’s lives. Secondly, it is needed in professional life such as teacher, administrator, and so on. Thirdly, it may be needed in conducting research in language study.

Testing or other forms of assessment is actually a learner’s need, too. It is also demanded by learners when they are learning as they need to know their own achievement (Ellaneri, 2009). In line with it, Harmer (2001:59) states that one of

3

the things that students expect from their teachers in learning English is an indication (which is represented by the tests’ result) of whether or not their English are right. In this case, according to him, teachers have to act as an assessor who can assess students’ English through some suitable ways, such as offering feedback and correction and grading students through testing their language proficiency.

Since language testing has significant roles in language education, therefore, there is no doubt that developing a critical understanding of the principles and practice of language testing is an undeniable need for all those involved in language educational practices. It means that adequate understanding of language testing is not merely relevant but also be needed for more generally those involved in using tests or the information they provide in practical and research contexts, and more specifically those actually involved in teaching language and creating language tests. Based on those arguments, it seems reasonable for English teachers and educational institutions to have sustainable capacity to develop and resemble language testing for students in their language teaching and learning programs.

As a result, teachers as the pillars of teaching practices are those who are central in developing and conducting a test as one of the teacher’s roles in educational duty is taking part as an assessor (Harmer, 2001:59). In the context of Indonesia, however, it is common to find erroneous English tests especially those which are constructed by the teachers (Wijanarko, 2004).

4

In consideration of the importance of language testing in language education and the fact that it is common to find erroneous English tests especially those which are constructed by the teachers, the present study, therefore, concerns with the importance of the practice of developing English tests. It investigates descriptively how final tests on English for the sixth grades primary school students in 2008/2009 are constructed by the teachers. It also reveals the test’s compliance with the objectives of the program. This study firstly focuses on the steps in the construction of the English final tests which are conducted by the selected English teachers. Then, it analyzes the compliance of the English final tests constructed by the teachers and the objectives of the English program applied at the schools.

1.2. Objectives of the Study

This study is particularly attempted at investigating descriptively how the construction of the final test on English which are done by the English teachers for the sixth grades primary school students in 2008/2009 and how the test’s compliance with the objectives of the program. Based on the formulation of the research questions, therefore, the purpose of this study is more specifically formulated as follow:

1. To find what steps the teachers do in constructing the final tests on English for the sixth grades primary school students.

2. To find out to what extent the tests comply with the objectives of the English program conducted in the schools or not.

5 1.3. Research Questions

This study is attempted at answering these following research questions: 1. What steps do teachers do in constructing the English final tests for the sixth

grades primary school students?

2. To what extent do the English final tests comply with the objectives of the English program conducted in the school?

1.4. Significance of the Study

This study is particularly attempted at investigating descriptively what steps the teachers do in constructing the final tests on English for the sixth grades primary school students and to find out the tests’ compliance with the objectives of the program. It is hoped that this study can reveal any valuable information deals with the development of English test, which is gained from the particular case of the English final tests’ development.

Despite numerous studies regarding language testing on English as a foreign language teaching and learning being of different contexts, empirical research is still lacking on the investigation of the process of construction of language test of public examination for primary school, particularly in the area where the present study is conducted. It might be caused by the fact that English has been being taught in primary school as a relatively new subject since in the context of Indonesia.

The finding of this study is, therefore, hoped to have valuable remark to be taken into account especially by the English teachers as the decision makers in

6

conducting such sustainable English language testing at schools. It is also hoped to be useful for those who deal with language teaching and language testing, since one way to minimize the potential for negative washback on instruction is to change the way we test so that the characteristics of the test and test tasks correspond more closely to the characteristics of the instructional program (Bachman & Palmer, 1996).

Specifically, the significance of the study is formulated as follows:

1. For English teachers, the result of this study is hoped to be valuable information especially related to their consideration and effort in attempting any development of EFL testing for their teaching and learning activities. 2. For other researchers, this study is hoped to be beneficial information which

may function as additional information or possibly as such starting point for their own further researches on language testing.

3. For the government as the policy maker of the public education, this study is hoped to provide such valuable information to be taken into account especially by the Department of the National Education of Indonesia in developing and conducting such sustainable English language testing for students which does not only comply the objectives of the program and the students’ need, but also being able to motivate students and to support teachers to conduct such quality language testing activities for students at school.

7 1.5. Scope of the Study

To focus the study into its’ particular concern, the researcher makes some limitations of the scope of the study. The present study attempts to investigate descriptively what steps the teachers do in constructing the final tests on English for the sixth grades primary school students in 2008/2009 and to find out the tests’ compliance with the objectives of the English program. It deals only with the case of the construction of the English final tests for the students of the sixth grade primary school which is conducted by the teachers at the primary Schools in Purworejo, Central Java, in the academic years of 2008/2009.

The study involves only the sample which is quite limited as it involves three selected teachers who teach English in three different primary schools as the sample of the study. The three teachers are selected as the respondents since they are obliged by the UPTD (the Unit of Technical Operation of the Office of National Education) to construct the English tests for the sixth grade of elementary schools students. Due to the limitation of the research scope, as well as the time constraint, the data will be collected from the limited particular sample.

In relation to the second research question, the research particularly concerns only with the English final tests for the sixth grade primary school students which are in the form of written multiple choice tests. Therefore, the tests do not deal with language skills, but they deal with some discreet point of language elements, i.e. vocabulary and grammar. Consequently, the objectives of the English teaching and learning here, particularly deal with the standards of competence which also deals with the students’ mastery of language elements of

8

vocabulary and grammar which were involved in all of the semesters (from the first semester of the fifth grade to the second semester of the sixth).

Since the characteristic of qualitative research is subjective and it is an exploration of what is assumed to be a dynamic reality in which it does not claim that what is discovered in the process is universal and, thus, replicable (McKereghan, 1998), therefore the result of the study may not be generalized for other cases in other different contexts.

1.6. Definitions of the Terms

This present study operates some key terms which are in details explained in the following definitions:

a) Investigation

Investigation is the process of inquiring into a matter through research, follow-up, study, or formal procedure of discovery (online-dictionary). In this particular context of the study, investigations means inquiring the problems which are represented by two research questions through a research.

b) Language Test Construction

Language Test Construction is the process of developing a test for measuring learners’ achievement in a language program which involves several periodical steps. In this research, the language test construction means the construction of English final tests of the selected elementary

9

schools constructed by the teachers through steps who are the respondents of the study.

c) Compliance

Compliance is the degree of how something submits or conforms to its requirements, rules, or objectives. The word compliance is used in this research particularly in the sentence ‘the English final tests’ compliance with the objectives of the program’ to represent the degree of how the English final tests submits or conforms to the objectives of the English program conducted in the schools.

d) Program:

A program is a plan of what is to be done (Hornby, 1974:666), or a plan which has been developed for particular purposes or objectives (Collins’ English Dictionary). Here, the researcher uses the term program particularly in the phrase ‘English program’ which means systematic plan in the form of syllabuses of the curriculum which has been developed to achieve particular purposes or objectives.

e) English final test

English Final Test is an achievement test which is used for measuring students’ achievement of English materials addressed in curriculum and roles to determine whether course objectives have been met –and appropriate knowledge and skills acquired- by the end of period of instruction (Brown, 2004). In this context of study, the English final test is the achievement test on English which is used for measuring the sixth

10

grade elementary schools students’ achievement of English materials addressed in curriculum and roles to determine whether course objectives have been met –and appropriate knowledge and skills acquired- by the end of period of instruction of English teaching and learning activities in the three elementary schools which have been selected as the sites of the study. The test is limited in the form of written multiple choice test which deals with linguistic elements of the target language, i.e. vocabulary and grammar.

1.7. Organization of the Thesis

The report of this study is organized into five chapters. The first chapter mainly discusses the reason and the purposes of the study. It presents the background of the study, the aims of the study and research questions, significance of the study, the scopes of the study, the definitions of the terms, the delimitation of the study, and the thesis organization.

Some concepts adopted for the study is discussed on chapter two. The definition of language testing, the qualities of language testing, the understanding of achievement test on foreign language, and the process of constructing a language test are discussed in series. In addition, this chapter discusses the previous studies undertaken by others on similar issues.

11

The third chapter of this thesis explains the way the researcher carried out this study. It covers the research design, the research site, the participants, the data collection techniques, and the data analysis.

Chapter four is the data presentation and discussion. Here the data which are from questionnaires, interviews and documentations are categorized and analyzed based on the formulation of the research questions.

The last chapter consists of the conclusion of the study based on the results of the study and some suggestions which are possibly valuable for those who are interested in English language teaching and English language testing, especially for those who are the test writers for the final tests on English for the sixth year of elementary school students.

36 CHAPTER III

RESEARCH METHODOLOGY

To achieve the objectives of the study, the method used in this inquiry was descriptive qualitative. Dealing with such a method, in this chapter, the writer discusses the research design, the setting of the study, the participants of the study, the data gathering techniques, and the method of analyzing the obtained data.

3.1. Research Design

The present inquiry was considered as a research since it in nature involves elements of a research. According to Nunan (1992), a research is a systematic process of inquiry consisting of three elements; (1) question, problems or hypothesis, (2) data, and (3) analysis and interpretation of data. It is a process which involves defining problem, stating objectives, and formulating a hypothesis. In the other words, this study can be considered as a systematic process of inquiry which involves gathering information, classification, analysis, and interpretation to see to what extent the initial objective has been achieved.

Based on the nature, the purpose and research questions as stated in the previous chapter, this study was conducted through employing a qualitative research design, in the form of a descriptive case study. Creswell (1994) states

37

that qualitative research is an inquiry process of understanding a social or human problem, based on building a complex, holistic picture, formed with words, reporting detailed informants, and conducted in a natural setting. While Erickson (1986) as cited in Stainback and Stainback (1988) defines qualitative research as a research which involves (a) intensive, long term participation in a field setting, (b) careful recording of what happens in the setting by writing field notes and interview notes and by collecting other kinds of documentary evidence; (c) analytic reflection on the documentary records obtained in the field; and (d) reporting the results by means of detailed descriptions, direct quotes from interviews, and interpretative commentary.

According to the definitions above, it seems clear that qualitative research is an inquiry method which focuses on revealing the mental process and what are involved in the process regarding whole aspects related to the whole context in social world. Therefore, it deals with answering questions of how, what and why. Related to it, Merriam (1988) and Creswell (1994) give further explanation on qualitative research’s characteristic, that qualitative research is focused primarily on process, rather than outcomes or products. This kind of research is also descriptive in which the researcher is interested in process, meaning, and understanding gained through words or pictures.

In harmony with the conceptions above, this particular study was particularly attempting at revealing the process of how the teacher constructs the final test for the students in the end of the English program conducted at the primary school. It descriptively answered the question of how the teachers

38

constructed the final test on English for the sixth grades primary school students and also to find out the tests’ complies with the objectives of the English program conducted in the schools.

The process of this qualitative study was inductive in that the researcher builds abstractions, concepts, hypotheses, and theories from details. It was more interested in seeing how participant makes sense of his or her experiences, and the structures of the world. In this present study, the researcher was the primary instrument for data collection and analysis. Data were mediated through this human instrument, rather than through inventories, questionnaires, or machines.

The form of case study had been chosen for several reasons. First, this study were carried out in a single case (Stake in Emilia, 2005:8), that was, investigating descriptively how teacher develop a final test on English for the sixth grades primary school students in 2008/2009 and the test’s compliance with the objectives of the English program. Secondly, for conducting the present case study, the researcher was actually working in the nature of technique in doing a case study since it was considered as the most suitable method to reveal the questions of the study. Here, the researcher explored a single entity or phenomenon (as the case) bounded by time and activity; which was the process of how the English tests were being constructed by the participants.

The researcher collected detailed information through a variety of data collection procedures over a sustained period of time, which was conducted by conducting interview and documentation. This case study was a descriptive record of an individual's experiences and/or behaviors kept by the researcher, who was

39

an outside observer. Besides, this study employed multiple sources of evidence-converging from the same set of issues, which constituted the important aspect of a case study (Yin, 2003:85 in Ansori, 2007). It was also mentioned that multiple data collections also intended to expand the construct validity of the study (Merriam, 1998). The data collecting techniques operated in this study were questionnaire, interview and documentation.

3.2. Setting

The study focused on investigating descriptively how English teachers develop final tests on English for the sixth grades primary school students in 2008/2009 and the test’s compliance with the objectives of the English program. It was conducted at three primary schools in Purworejo, Central Java, and it involved three English teachers of the schools who were chosen purposively as the sample of the study. In consideration of normative ethical aspect of the study (Reese et.al. 1984, Martin 2009, Ekvall et.al 2005), therefore, the name of the chosen participants and the schools are not stated explicitly in this research report since it deals with individual actions and institutional works.

The site was chosen as it was apparently valuable to make the study to be more efficiently conducted, since both of the researcher and the participant lived in the same city, therefore the researcher possibly had adequate access to the participants. The researcher was also interested in conducting the study in the site since English had been known as relatively new subject for students of primary

40

schools in the area so that it seemed valuable to evaluate whether the construction of testing on English subject had been conducted properly there or in contrary.

3.3. Participants

Since this study was attempted at investigating descriptively how final tests on English for the sixth grades primary school students in 2008/2009 were constructed by the teachers and the test’s compliance with the objectives of the English program, therefore, this study involved English teachers as the participants of the study. There were three English teachers of three different primary schools in Purworejo, Central Java, who had been teaching English and constructing English tests for the final examination, who were chosen purposively as the participants of this study.

There were several reasons for choosing the teachers as the respondent of the study. First, the teachers had been experiencing teaching English in the classes of the primary schools for years and that they were developing the English final test for the sixth grade primary school students by their selves according to the objectives of the program. Secondly, the teachers were considered educated, and therefore, had knowledge, adequate perception and practice on the identified case. Thirdly, the teachers, as the respondents, were accessible. It enabled the researcher to conduct the study efficiently. And finally, the participants involved were considered adequate to fulfill the need for conducting the study.

41 3.4. Data Collection Techniques

There were two instruments used for collecting the data of this study. Those instruments were: interview and document analysis.

This present study focused on understanding and revealing the process of constructing the test as the ‘world’ being investigated from the subject’s point of view. The process of constructing the test was investigated through operating interview with the subject. It was the characteristic of qualitative research which apply interview in inquiry. In this qualitative research, the interview was conducted to understand the world from the subject’s point of view, to unfold the meaning of peoples’ experiences, to uncover their lived world prior to scientific explanation (Kvale, 1996:1).

The interview in the present study was aimed at revealing the process of how the teacher constructs the test. It focused on the steps in constructing the test. As stated in chapter II, there should be certain periodic steps in constructing a quality language test in this process. The method of obtaining the data was operated for this case since the process of constructing the test, like writing process, seemed to be psychological process which was invisible and, therefore, revealing the process could be conducted by interviewing the subject.

Interview, in this study, meant a conversation between the researcher and the respondents, i.e. the selected English teachers. The interview was the conversation with a purpose (Merriam, 1988:71). In interview, one gives information to another. In this case, information given by the respondents was the information needed by the researcher as the data of the research.

42

Interview was considered very effective in this study. Guba and Lincoln (1981:158) as quoted by Ansori (2007) said that:

Experience in the field demonstrates that on some occasions, individual, one-on-one interviewing is effective, that panel interviewing can sometimes elicit about as much information as it is possible to gamer from a particular group, and that dual-interviewer modes may prove extraordinarily useful under some circumstances.

In conducting the interview, the individual, open-ended interview technique was applied. In this study, the researcher interviewed the teachers as the respondents in March to June 2009.

The interviews were then documented by voice recorder and then they were transcribed in the form of written texts. All the texts were read by the respondent to confirm that all data are correct. Therefore, some correction or clarification occurred during this process. It meant that checking was needed to develop the credibility and accuracy of the data (Alwasilah, 2004).

Document analysis was the second technique in collecting the data of the study which was used to examine the tests’ correspondence with the objectives of the program. This data gathering technique was used to obtain the data in the form of documents. The first thing to do is to get the complete information of the objectives of the English program implemented at the primary school and the written English final tests in the sixth grade of the primary school, the researcher used the document analysis technique of obtaining the data. It then is followed by analyzing the tests batteries and correspond them in terms of their particular

43

objectives so that the researcher may find the tests’ correspondence with the objectives of the English teaching program conducted in the classes.

The steps of collecting the data by using document analysis technique were as follows:

1. Getting the document of the objectives of the English program implemented in the primary school, i.e. the curriculum and the syllabus of English program.

2. Making a copy of the curriculum and syllabus of the program.

3. Getting the printed documents of the written English test batteries used for the final test for the sixth grade primary school students in the academic year of 2008/2009.

4. Making a copy of the documents as the data.

3.5. Data Analysis

Data analysis was the process of systematically searching and arranging the information of the data gathering techniques (the interview transcripts, and also the document analysis) that were accumulated to increase understanding of them and to enable to present what researcher had discovered to others (Bogdan and Biklen, 1992:153 in Ansori, 2007). In this study, the data analysis involved working with data, organizing them, breaking them into manageable units, synthesizing them, searching for patterns, discovering what was important and

44

what was to be learned, and deciding what would be told to others in the report of the study, according to the two determined research questions.

In doing data analysis of the data obtained from the interview, the result of the interview was firstly transcribed. In the transcription, similar with the data of the questionnaires, the researcher used the initial name of the participants to keep the objectivity in analyzing the further data (Kvale, 1996). The interviews which had been transcribed were then re-checked by each of the respondents to ensure that the transcriptions really reflected what the participants had meant (Kvale, 1996), and to enable the researcher to get some possible input from the respondents (Smith et.al, 1990).

The examined transcriptions were then categorized according the topic to which they were addressed for. This categorization were then continued by interpreting and analyzing the data; whether the revealed process of constructing the test were correspond with the theory proposed by the language testing experts, and revealing what possible factors caused it and why. The analysis, then, were reported descriptively.

The analysis of the documents data, on the other hand, involved the following stages:

1. Classifying the objectives of the English program according to the type of language skills (listening, speaking, reading and writing) and the type of language elements (vocabulary, grammar, and pronunciation).

45

2. Classifying the English final test batteries according to the type of language skills (listening, speaking, reading and writing) and the type of language elements (vocabulary, grammar, and pronunciation).

3. Comparing the objectives of the English program and the test batteries according to the type of language skills (listening, speaking, reading and writing) and the type of language elements (vocabulary, grammar, and pronunciation).

4. Interpreting the result of the analysis by giving description and some critical comment on each of the case.

108 CHAPTER V

CONCLUSSIONS AND SUGGESTIONS

In this chapter, the writer elaborates the concluding remarks of the important findings mentioned in the previous chapter’s discussions as the answer to the research problems. This chapter consists of two parts. The first part is the conclusion of the study, while the second is the recommendation which possibly valuable for those who are interested and involved in the English language test construction activities.

5.1. Conclusions

Based on the discussion in the previous chapter, it is found that the respondents’ activities in constructing the English final test for elementary schools’ students did not completely perform each step of constructing an English test in the theory stated by Harris (1969). It was shown by the absence of the step of pre-testing the material and analyzing the results in the series of activities conducted by all of the respondents.

There were some reasoning arguments on why they did not conduct pre-test of the formulated pre-test material and analyzing the results of the pre-pre-test. All of the respondents agreed in stating that the time constraint was the cause of it. They also said that performing a pretest required facilities such as test-schedule,

109

proctors, fund, and students as the test-takers. This condition was related to the policy maker of the schools. Consequently, it seems valuable for not only English teachers but also the principals of schooling institutions who need to develop right understanding of the principles and practice of language testing, including on how an English test should be constructed properly.

The document analysis of the syllabuses which represents the objectives of the English program (which deals with language elements of vocabulary and grammatical structures) and the tests items shows that not all of the items of the tests constructed by the respondents were in relevance to the contents of the syllabuses. The erroneous items of the tests failed to achieve the objectives of some tests items to test what they should really test.

The existence of the erroneous items in the tests proofs that the function of re-reading the constructed tests items plays important role in developing quality test which has no error. Indeed, the absence of pre-testing steps in the test construction, as has been discussed in the previous analysis of the tests construction, also caused the erroneous items unrevealed. It is in line with the theory that careful review will often identify items which otherwise would be lost later in pretesting or would arouse the criticism of the subject matter specialists were they to appear in the final version of the test (Harris, 1969:103). The framework of content validity proposed by Brown (2004) and of Bachman and Palmer (1996:356) are also supported by the evidence that the correspondence between the tests materials and the contents of the program which was shown in

110

the previous discussion above was able to be achieved by the construction of the tests which were based on the scope of materials presented in the syllabuses.

5.2. Suggestions

Based on the conclusion of the study above, there are some suggestion which possibly valuable for those who are interested and even involved in constructing English language test.

For English teachers, developing understanding on the principles and practices of language testing especially on construction of language testing is an undeniable need since it is a fundamental conception which will determine whatever the teachers do in developing or constructing a language test. Following the stages which are proposed by some language testing experts are so valuable in determining the quality of the test resulted.

For the schools as the educational institutions, facilitating the process of constructing a language test conducted by teachers is also an undeniable need. As all of the respondents implied that one of the reasons they had difficulties in performing English test construction completely was the absence of support and facility from the schools as the institutions, therefore, the commitment of the teachers and the principals are essential in determining the further quality operational or practical policies in facilitating the test construction on English program conducted at the schools.

111 REFERENCES

Alderson, C.J., Clapham, C. and Wall, D. (1995). Language Test Construction and Evaluation. England: Cambridge University Press.

Alwasilah, C.A. (2003). Perspektif Pendidikan Bahasa Inggris di Indonesia dalam Konteks Persaingan Global. Bandung: CV Andira Press.

---. (2003). Pokoknya Kualitatif: Dasar-Dasar Merancang dan Melakukan Penelitian Kualitatif, Jakarta: Pustaka Jaya.

Ansori, A. (2007). The Process of Writing: A Case Study of Post Graduate Students’ Academic Writing (a thesis). Unpublished.

Bachman, FL. and Palmer, SA.. (1996). Language Testing in Practice: Designing and Developing Useful Tests. Oxford: Oxford University Press.

Bachman, FL. (1990). Fundamental Considerations in Language Testing. England: Oxford University Press.

Bartz, Walter H. (1979). Language in Education: Theory and Practice. United States of America: Centre for Applied Linguistics, Clearinghouse on Language and Linguistics.

Berglund, E and Eriksson, Marten.. (2000). Reliability and content validity of a new instrument for assessment of communicative skills and language abilities in young Swedish children. Logopedics Phoniatrics

112 Accessed on 1 May 2009 on

http://www.informaworld.com/smpp/content~db=all~content=a71378 9152

Brown, H. Douglas. (2001). Teaching by Principles: An Interactive Approach to Language Pedagogy. New York: Pearson Education Company.

---. (2004). Language Assessment: Principles and Classroom Practices. New York: Longman Pearson Education Inc.

Chappele, C.and Dan, D. (1993). A New Decade of Language Testing Research: Selected Papers from the 1990 Language Testing Reseach

Colloquium. United States of America: Teachers of English to Speakers of Other Language (TESOL) Inc.

Chappelle, C.A. and Jamieson, Joan. Three Challenges in English Language Assessment. Accessed on 1 May 2009 on

http://www.pearsonlongman.com/ae/multimedia/pdf/Carol_Joan_PDF .pdf

Connole, H., Smith, B., & Wiseman,R. (1993). Research Methodology: Issues and Methods in Research. Melbourne: Deakin University Press.

Creswell, J. W. (1994). Research design: Qualitative & quantitative approaches. Thousand Oaks, CA: SAGE Publications Inc.

---. (1998). Qualitative Inquiry and Research Design: Choosing among Five Traditions. United States of America: SAGE Publications Inc.

113

Cronbach, Lee Joseph. (1984). Essentials of Psychological Testing. Cambridge: Harper and Row Publication Inc.

Crystal, David. (1991). A Dictionary of Linguistics and Phonetics –Third Edition-. United States of America: Basil Blackwell Ltd.

Dictionary.com. (Accessed on August 01, 2009). http://dictionary.reference.com/ browse/program

Ekvall, T. et.al. (2005). Normative Ethics and Methodology for Life Cycle Assessment. Accessed in July 2nd 2009 on

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VF

X-4GFV5H5-1&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&_docanchor= &view=c&_searchStrId=946764764&_rerunOrigin=google&_acct=C 000050221&_version=1&_urlVersion=0&_userid=10&md5=8c3b3e3 0e65c87a38c12b33177f250b6

Ellaneri, K. (2009). Area of problems in Writing Recount Text. Jurnal Pendidikan Inovatif. Accessed on January 2009 on

http://jurnaljpi.wordpress.com/2009/01/02/elanneri-karani/#more-46

Grabowsky, K.C. Investigating the Construct Validity of a Performance Test Designed to Measure Grammatical and Pragmatic Knowledge. Accessed on 21 May, 2009 on

http://www.lsa.umich.edu/UMICH/eli/Home/Research/Spaan%20Fell owship/pdfs/SpaanV6Grabowski.pdf

Gronlund, E.,N. (1982). Constructing Achievement Test. United States of America: Prentice Hall Inc.

114

Harmer, Jeremy. (2001). The Practice of Eglish Language Teaching. England: Longman.

Harris, P. David. (1969). Testing English as a Second Language. United States of America: McGraw-Hill, Inc.

Hornby, A.S, (2002) Oxford Advanced Learner’s Dictionary, New York: Oxford University Press.

Hughes, Arthur. (1989). Testing for Language Teachers. Cambridge: Cambridge University Press.

Iwamura, Mitsuru, Machiya, M., Takashi, T. and Takashi, T. (2006). A Report on Improvements in English Education for Engineers in the Age of Globalization at Hachinohe Institute of Technology [in Japanese]. A Journal. Accessed on August 1st 2009 on

http://nels.nii.ac.jp/els/110004616426.pdf?id=ART0007328491&type =pdf&lang=en&host=cinii&order_no=&ppv_type=0&lang_sw=&no= 1250733753&cp=

Kvale, Steinar. (1996). Interviews: An Introduction to Qualitative Research Interviewing. California: SAGE Publications Inc.

Liao, Y F. Issues of Validity and Reliability in Second Language Performance Assessment. Teachers College, Columbia University Working Papers in TESOL & Applied Linguistics, Vol. 4, No. 2. Accessed on 21 May, 2009 on

http://www.tc.columbia.edu/academic/tesol/WJFiles/pdf/YenFenForu m.pdf

115

Martin, DG. (2009). Bureacratizing Ethics: Institutional Review Boards and Participatory Research. Accessed in July 1st 2009 on

http://www.acme-journal.org/vol6/DGM.pdf

Marzano, R.J., Pickering, D.J., and Mctighe, J. (1993). Assessing Students Outcomes: Performance AssessmentUusing the Dimensions of Learning Model. United States of America: Association for Supervision and Curriculum Development (ASCD).

McKereghan, L. Donna. (1998). Quantitative versus Qualitative Research: An Attempt to Clarify the Problem.

http://home.comcast.net/~lady.socrates/qvq.html

McNamara, Tim. (2000). Language Testing. United Kingdom: Oxford University Press.

Merriam, B.S. (1988). Case Study Research in Education: A Qualitative Approach. United States of America: Jossey-Bass Publishers.

Nasution, D.N. et.al, (2009). Pemanfaatan Multimedia untuk Menstimulus Imajinasi Penyusunan Kalimat Possesive Pronouns. Jurnal Pendidikan Inovatif. Accessed on 12th July 2009 on

http://jurnaljpi.wordpress.com/2009/01/02/anik-sri-mulyani-dinda-nauli-nasution-dan-maya-yuliantini/#more-47

Nunan, David. (1992). Research Methods in Language Learning. England: Cambridge University Press.

116

Purwanto, Ngalim. M. (1984). Prinsip-prinsip Dasar dan Teknik

EvaluasiPengajaran. Bandung: Penerbit PT. Remaja Rosdakarya Bandung.

Rausch, Anthony. (2000). Improvement in English Education from a Learning Perspective: The Teacher's View versus The Student's View. A Journal accessed on July 2009 on

http://www.jalt-publications.org/tlt/articles/2000/06/rausch

Reese, HW et.al. (1984). Normal and Normative Ethichs in Behavioral Sciences. Accessed in May 12th 2009 on

http://www.find-health- articles.com/rec_pub_11644156-normal-normative-ethics-behavioral-sciences.htm

Richard, Jack; John Platt; and Heidi Weber. (1985). Longman Dictionary of Applied Linguistics. England: Longman Group Limited.

Smith, B. et.al. (1990). Issues and Methods in Research: Study Guide. Australia: South Australian College of Advanced Education.

Stainback, S. & Stainback, W. (1988). Understanding and Conducting Qualitative Research. Virginia: The Council for Exceptional Children.

Townsend, Jane S.et.al. (1997). Writing Assessment: Multiple Perspectives, Multiple Purposes. Accessed on January 2nd 2009 on

http://find.galegroup.com/gps/retrieve.do?contentSet=IAC-Documents&resultListType=RESULT_LIST&qrySerId=Locale(en%2 C%2C)%3AFQE%3D(ke%2CNone%2C16)language+testing%24&sg HitCountType=None&inPS=true&sort=DateDescend&searchType=B asicSearchForm&tabID=T002&prodId=IPS&searchId=R1&currentPo

117

sition=33&userGroupName=ptn071&docId=A53877582&docType=I AC&contentSet=IAC-Documents

Walsh, W.Bruce and Nancy E.Bets. (1985). Test and Assessment. Eaglewood Cliffs. New jersey: Prentice Hall Inc.

Widoyoko, P.E.S, (2008). Analisis Kualitatif dalam Penelitian Sosial. http://www.um-pwr.ac.id/publikasi/13/analisis-kualitatif-dalam-penelitian-sosial

Wijanarko, Y. (2007). Using Stick Figure In ‘CTL’ to Improve Students’ Ability in Writing Recount Text Of Class VII-A SMPN 1 Lumajang. Jurnal Pendidikan Inovatif. Accessed on 1st May 2009 on

http://jurnaljpi.wordpress.com/2007/11/14/yusuf-wijanarko/#more-3

Wiseman, R. (1990). Issues and Method in Research. Study Guide. . South Australian College of Advanced Education.

Yin, R. K. (1993). Application of Case Study Research. Newsburry Park, California: SAGE Publication Inc.

112 Accessed on 1 May 2009 on

http://www.informaworld.com/smpp/content~db=all~content=a71378 9152

Brown, H. Douglas. (2001). Teaching by Principles: An Interactive Approach to

Language Pedagogy. New York: Pearson Education Company.

---. (2004). Language Assessment: Principles and Classroom

Practices. New York: Longman Pearson Education Inc.

Chappele, C.and Dan, D. (1993). A New Decade of Language Testing Research: Selected Papers from the 1990 Language Testing Reseach

Colloquium. United States of America: Teachers of English to

Speakers of Other Language (TESOL) Inc.

Chappelle, C.A. and Jamieson, Joan. Three Challenges in English Language

Assessment. Accessed on 1 May 2009 on

http://www.pearsonlongman.com/ae/multimedia/pdf/Carol_Joan_PDF .pdf

Connole, H., Smith, B., & Wiseman,R. (1993). Research Methodology: Issues

and Methods in Research. Melbourne: Deakin University Press.

Creswell, J. W. (1994). Research design: Qualitative & quantitative approaches. Thousand Oaks, CA: SAGE Publications Inc.

---. (1998). Qualitative Inquiry and Research Design: Choosing

among Five Traditions. United States of America: SAGE Publications

113

Cronbach, Lee Joseph. (1984). Essentials of Psychological Testing. Cambridge: Harper and Row Publication Inc.

Crystal, David. (1991). A Dictionary of Linguistics and Phonetics –Third Edition-.

United States of America: Basil Blackwell Ltd.

Dictionary.com. (Accessed on August 01, 2009). http://dictionary.reference.com/ browse/program

Ekvall, T. et.al. (2005). Normative Ethics and Methodology for Life Cycle Assessment. Accessed in July 2nd 2009 on

http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VF

X-4GFV5H5-1&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&_docanchor= &view=c&_searchStrId=946764764&_rerunOrigin=google&_acct=C 000050221&_version=1&_urlVersion=0&_userid=10&md5=8c3b3e3 0e65c87a38c12b33177f250b6

Ellaneri, K. (2009). Area of problems in Writing Recount Text. Jurnal Pendidikan Inovatif. Accessed on January 2009 on

http://jurnaljpi.wordpress.com/2009/01/02/elanneri-karani/#more-46

Grabowsky, K.C. Investigating the Construct Validity of a Performance Test Designed to Measure Grammatical and Pragmatic Knowledge.

Accessed on 21 May, 2009 on

http://www.lsa.umich.edu/UMICH/eli/Home/Research/Spaan%20Fell owship/pdfs/SpaanV6Grabowski.pdf

Gronlund, E.,N. (1982). Constructing Achievement Test. United States of America: Prentice Hall Inc.

114

Harmer, Jeremy. (2001). The Practice of Eglish Language Teaching. England: Longman.

Harris, P. David. (1969). Testing English as a Second Language. United States of America: McGraw-Hill, Inc.

Hornby, A.S, (2002) Oxford Advanced Learner’s Dictionary, New York: Oxford University Press.

Hughes, Arthur. (1989). Testing for Language Teachers. Cambridge: Cambridge University Press.

Iwamura, Mitsuru, Machiya, M., Takashi, T. and Takashi, T. (2006). A Report on Improvements in English Education for Engineers in the Age of Globalization at Hachinohe Institute of Technology [in Japanese]. A Journal. Accessed on August 1st 2009 on

http://nels.nii.ac.jp/els/110004616426.pdf?id=ART0007328491&type =pdf&lang=en&host=cinii&order_no=&ppv_type=0&lang_sw=&no= 1250733753&cp=

Kvale, Steinar. (1996). Interviews: An Introduction to Qualitative Research Interviewing. California: SAGE Publications Inc.

Liao, Y F. Issues of Validity and Reliability in Second Language Performance

Assessment. Teachers College, Columbia University Working Papers

in TESOL & Applied Linguistics, Vol. 4, No. 2. Accessed on 21 May, 2009 on

http://www.tc.columbia.edu/academic/tesol/WJFiles/pdf/YenFenForu m.pdf

115

Martin, DG. (2009). Bureacratizing Ethics: Institutional Review Boards and Participatory Research. Accessed in July 1st 2009 on

http://www.acme-journal.org/vol6/DGM.pdf

Marzano, R.J., Pickering, D.J., and Mctighe, J. (1993). Assessing Students Outcomes: Performance AssessmentUusing the Dimensions of

Learning Model. United States of America: Association for

Supervision and Curriculum Development (ASCD).

McKereghan, L. Donna. (1998). Quantitative versus Qualitative Research: An Attempt to Clarify the Problem.

http://home.comcast.net/~lady.socrates/qvq.html

McNamara, Tim. (2000). Language Testing. United Kingdom: Oxford University Press.

Merriam, B.S. (1988). Case Study Research in Education: A Qualitative

Approach. United States of America: Jossey-Bass Publishers.

Nasution, D.N. et.al, (2009). Pemanfaatan Multimedia untuk Menstimulus Imajinasi Penyusunan Kalimat Possesive Pronouns. Jurnal Pendidikan Inovatif. Accessed on 12th July 2009 on

http://jurnaljpi.wordpress.com/2009/01/02/anik-sri-mulyani-dinda-nauli-nasution-dan-maya-yuliantini/#more-47

Nunan, David. (1992). Research Methods in Language Learning. England: Cambridge University Press.

116

Purwanto, Ngalim. M. (1984). Prinsip-prinsip Dasar dan Teknik

EvaluasiPengajaran. Bandung: Penerbit PT. Remaja Rosdakarya

Bandung.

Rausch, Anthony. (2000). Improvement in English Education from a Learning

Perspective: The Teacher's View versus The Student's View. A Journal

accessed on July 2009 on http://www.jalt-publications.org/tlt/articles/2000/06/rausch

Reese, HW et.al. (1984). Normal and Normative Ethichs in Behavioral Sciences.

Accessed in May 12th 2009 on

http://www.find-health- articles.com/rec_pub_11644156-normal-normative-ethics-behavioral-sciences.htm

Richard, Jack; John Platt; and Heidi Weber. (1985). Longman Dictionary of

Applied Linguistics. England: Longman Group Limited.

Smith, B. et.al. (1990). Issues and Methods in Research: Study Guide. Australia: South Australian College of Advanced Education.

Stainback, S. & Stainback, W. (1988). Understanding and Conducting Qualitative Research. Virginia: The Council for Exceptional Children.

Townsend, Jane S.et.al. (1997). Writing Assessment: Multiple Perspectives, Multiple Purposes. Accessed on January 2nd 2009 on

http://find.galegroup.com/gps/retrieve.do?contentSet=IAC-Documents&resultListType=RESULT_LIST&qrySerId=Locale(en%2 C%2C)%3AFQE%3D(ke%2CNone%2C16)language+testing%24&sg HitCountType=None&inPS=true&sort=DateDescend&searchType=B asicSearchForm&tabID=T002&prodId=IPS&searchId=R1&currentPo

117

sition=33&userGroupName=ptn071&docId=A53877582&docType=I AC&contentSet=IAC-Documents

Walsh, W.Bruce and Nancy E.Bets. (1985). Test and Assessment. Eaglewood Cliffs. New jersey: Prentice Hall Inc.

Widoyoko, P.E.S, (2008). Analisis Kualitatif dalam Penelitian Sosial.

http://www.um-pwr.ac.id/publikasi/13/analisis-kualitatif-dalam-penelitian-sosial

Wijanarko, Y. (2007). Using Stick Figure In ‘CTL’ to Improve Students’ Ability in

Writing Recount Text Of Class VII-A SMPN 1 Lumajang. Jurnal

Pendidikan Inovatif. Accessed on 1st May 2009 on

http://jurnaljpi.wordpress.com/2007/11/14/yusuf-wijanarko/#more-3

Wiseman, R. (1990). Issues and Method in Research. Study Guide. . South Australian College of Advanced Education.

Yin, R. K. (1993). Application of Case Study Research. Newsburry Park, California: SAGE Publication Inc.