A study on washback of English national examination among 12th graders of a private religion-based vocational High School in Yogyakarta.

(1)

ABSTRACT

Subagyo, Anggun Wahyu Purnomo Sidi. (2014). A Study on Washback of English National Examination among 12th_{Graders of a Private Religion-based} Vocational High School in Yogyakarta. Yogyakarta: Sanata Dharma University.

Indonesia implements national examination (NE) as one of the considerations for deciding students’ graduation from secondary schools. As a high-stakes testing, NE has inevitably given washback. The washback appears to impact students, teachers, and school during the NE preparation. The washback itself can be positive or even negative. It depends on the nature of the response shown by the parties. Thus, the researcher was interested to study what washback happens to the parties and how it can happen.

This study has two research problems. First, it is 1) What washback do the twelfth graders of a private religion-based vocational high school in Yogyakarta experience in response to National Examination preparation? Second, it is 2) How does the washback happens to the students, teachers, and the school? Therefore, the objectives of this study are 1) to find what washback happens to the parties and 2) how the washback happens to the parties.

This study is qualitative research. It used case study method. The data was obtained from nine participants consisted of seven twelfth graders, an English teacher, and a vice principal of a private religion-based vocational high school in Yogyakarta. The researcher obtained the data through observation and interviews. To solve the first research problem, the researcher employed some theories from some educationists to comprehend the concept of washback. To solve the second research problem, the researcher used some theories from some educationalist as well to comprehend and picture of how washback happen.

The data in this research is reflection and interview record. The researcher conducted inductive analysis to investigate the data. Afterwards, the researcher presented the findings and discussions deductively.

This study supports learning, teaching, and school management, especially about preparing the NE. Secondary school students become more aware of the impacts of NE. Besides, it also encourages them to be more prepared with the potential challenges such as feeling of pressure and time constraints. For teachers, they can comprehend the phenomenon faced by their students. Their comprehension facilitates them to find the best way to engage with the situation. For future researchers, this research inspires those who are interested in conducting research about washback.

Keywords: washback, national examination, vocational high school, inductive inquiry, deductive narrative

(2)

ABSTRAK

Indonesia menerapkan ujian nasional (UN) sebadai salah satu penentu kelulusan para siswa di jenjang sekolah menengah. Sebab UN sangat menentukan, terjadinya washback menjadi tidak terelakkan. Washback memberikan dampak kepada siswa, guru, dan sekolah selama persiapan UN. Washback sendiri dapat menguntungkan ataupun merugikan. Oleh karena itu, peneliti tertarik untuk meneliti washback apa yang terjadi dan bagaimana washback itu terjadi.

Penelitian ini mempunyai dua pertanyaan, yaitu 1) Washback apa yang dialami oleh para siswa kelas dua belas di sebuah sekolah menengah kejuruan keagamaan di Yogyakarta dalam menanggapi persiapan UN? dan 2) Bagaimana

washback tersebut dapat terjadi pada para siswa, para guru, dan sekolah? Maka dari itu, tujuan dari penelitian ini adalah 1) menemukan macam washback yang dialami oleh para siswa kelas dua belas di sebuah sekolah menengah kejuruan keagamaan di Yogyakarta dalam menanggapi persiapan UN dan 2) mengungkap mekanisme terjadinya washback pada siswa, guru, dan sekolah.

Penelitian ini adalah penelitian kualitatif studi kasus. Data diperoleh dari sembilan responden yang terdiri dari tujuh siswa kelas dua belas, seorang guru bahasa Inggris, dan seorang wakil kelapa sekolah dari sebuah sekolah menengah kejuruan keagamaan di Yogyakarta. Data dikumpulkan dari observasi dan wawancara. Jawaban pertanyaan pertama didapat dari memanfaatkan beberapa teori dari para pakar pendidikan untuk memahami konsep tentang washback.

Pertanyaan ke dua dijawab menggunakan beberapa teori dari para pakar pendidikan untuk memahami dan menggambarkan proses terjadinya washback.

Data penelitian ini berupa refleksi dan hasil wawancara. Peneliti melakukan analisis induktif untuk meneliti data. Seletah itu, peneliti menampilkan penemuan dan pembahasan secara deduktif.

Penelitian ini mendukung pembelajaran, pengajaran, dan pengelolaan sekolah khususnya tentang mempersiapkan UN. Para siswa sekolah menengah menjadi waspada tentang dampak UN. Selain itu, mereka menjadi terdorong untuk lebih mempersiapkan diri dengan tantangan UN seperti perasaan tertekan dan waktu persiapan yang singkat. Bagi para guru, mereka akan mendapatkan pemahaman tentang fenomena yang dihadapi siswa-siswanya. Pemahaman tersebut membantu mereka untuk menemukan cara terbaik untuk menyikapi situasi tersebut. Untuk para peneliti lain, penelitian ini memberikan inspirasi bagi mereka yang tertarik untuk membuat penelitian tentang washback.

Keywords: washback, national examination, vocational high school, inductive inquiry, deductive narrative

(3)

A STUDY ON WASHBACK OF ENGLISH NATIONAL

EXAMINATION AMONG 12

_{GRADERS OF A PRIVATE}

RELIGION-BASED VOCATIONAL HIGH SCHOOL IN

YOGYAKARTA

A SARJANA PENDIDIKAN THESIS

Presented as Partial Fulfillment of the Requirements to Obtain the Sarjana Pendidikan Degree

in English Language Education

Anggun Wahyu Purnomo Sidi Subagyo Student Number: 101214180

ENGLISH LANGUAGE EDUACATION STUDY PROGRAM DEPARTMENT OF ENGLISH LANGUAGE AND ARTS EDUCATION

FACULTY OF TEACHERS TRAINING AND EDUCATION SANATA DHARMA UNIVERSITY

YOGYAKARTA 2014

(4)

A STUDY ON WASHBACK OF ENGLISH NATIONAL

EXAMINATION AMONG 12

_{GRADERS OF A PRIVATE}

RELIGION-BASED VOCATIONAL HIGH SCHOOL IN

YOGYAKARTA

A SARJANA PENDIDIKAN THESIS

Presented as Partial Fulfillment of the Requirements to Obtain the Sarjana Pendidikan Degree

in English Language Education

Anggun Wahyu Purnomo Sidi Subagyo Student Number: 101214180

ENGLISH LANGUAGE EDUACATION STUDY PROGRAM DEPARTMENT OF ENGLISH LANGUAGE AND ARTS EDUCATION

FACULTY OF TEACHERS TRAINING AND EDUCATION SANATA DHARMA UNIVERSITY

YOGYAKARTA 2014

(5)

(6)

(7)

Trust in the Lord with all thine heart ;

and lean not unto thine own understanding.

In all thy ways acknowledge him,

and he shall direct thy paths.

Proverbs 3:5-6

I dedicate this thesis to the Almighty Jesus Christ,

my parents, my sister,

(8)

(9)

(10)

vii ABSTRACT

Keywords: washback, national examination, vocational high school, inductive inquiry, deductive narrative

(11)

viii ABSTRAK

Pertanyaan ke dua dijawab menggunakan beberapa teori dari para pakar pendidikan untuk memahami dan menggambarkan proses terjadinya washback.

Data penelitian ini berupa refleksi dan hasil wawancara. Peneliti melakukan analisis induktif untuk meneliti data. Seletah itu, peneliti menampilkan penemuan dan pembahasan secara deduktif.

Keywords: washback, national examination, vocational high school, inductive inquiry, deductive narrative

(12)

ACKNOWLEDGEMENTS

First and foremost, I would like to greatly thank Lord Jesus Christ for His great blessings in my life. He gives me health, spirit, and strenght to finish my study and to arrive at another milestone of my life.

I would like to express my deepest gratitude to my advisor, Markus Budiraharjo, M.Ed., Ed.D. for generously guiding and assisting me during the process, making himself available for consultations anytime and anywhere, and giving attention and suggestion until the final process of writing this thesis. It is a great blessing for me to have a super kind advisor like him. I specially thank Barli Bram, Ph.D. for helping me to check my grammar. I got precious suggestions even though it was only one meeting. I also specially thank my academic advisor, Henny Herawati, M.Hum. I thank all lecturers of English Language Education Study Program for lecturing along the study in Sanata Dharma University. I would also thank all secretariat staff of English Language Education Study Program and Faculty of Teachers Training and Education of Sanata Dharma University and all librarians for giving the best and friendly service.

I would like to thank my family especially my parents, Benedictus Bagyo and Mukidah, who always pray for me, encourage, and take care of me in finishing my study. I sincerely address my special gratitude to all friends in PBI. I thank PBI 2010, especially Class C. They always cheers me up when I was stuck doing my thesis. I thank Ardhi, Dea,and Bayu who give corrections and suggestions for my writing. I deliver my special appreciation to Chandra “Ahong”, Pandhu, Tiwi,

(13)

Disa, Nana, Nutnut, Ineke, Amel, Lina “Makibo”, Nadia “Ijah”, Estu

“Kapuk”, Ratri, Sunu, Edo,Ega “Gaplek”, Veni, Yona, Nono, Ria, and Chaca. The last, I would like to thank all who have support and help me to finish my thesis. May God bless them always.

(14)

TABLE OF CONTENTS

page

TITLE PAGE ... i

APPROVAL PAGE ... ii

DEDICATION PAGE ... iv

STATEMENT OF WORK’S ORIGINALITY ... v

LEMBAR PERNYATAAN PERSETUJUAN PUBLIKASI ... vi

ABSTRACT ... vii

ABSTRAK ... viii

ACKNOWLEDGEMENTS ... ix

TABLE OF CONTENTS ... xi

LIST OF TABLES ... xv

LIST OF FIGURE ... xvi

LIST OF APPENDICES ... xvii

CHAPTER I. INTRODUCTION A. Research Background ... 1

B. Research Problems ... 5

C. Problem Limitation ... 5

D. Research Objectives ... 6

E. Research Benefits ... 6

1. For Students ... 6

2. For English Teachers... 6

3. For School ... 7

4. For Future Researchers ... 7

F. Definition of Terms... 7

1. Washback ... 7

2. National Examination... 8

3. Vocational High School ... 8

(15)

xii

5. Deductive Narrative ... 8

CHAPTER II. REVIEW ON RELATED LITERATURE A. Theoretical Description ... 10

1. Exploring the Concept of Washback... 10

a. The Origin of Washback ... 10

b. The Definition and Scope of Washback... 12

c. Washback Defined in this Strudy ... 15

1) Negative Washback ... 16

2) Positive Washback ... 17

d. The Function of Washback ... 18

2. Exploring the Research Phenomenon ... 22

a. Influences of High-stakes Testing ... 22

b. Measurement-driven Instruction ... 24

3. Washback within Vocational High School Context... 27

B. Theoretical Framework ... 28

CHAPTER III. RESEARCH METHODOLOGY A. Research Method ... 30

B. Research Setting ... 31

C. Research Participants... 32

1. First Student ... 32

2. Second Student ... 33

3. Third Student ... 34

4. Fourth Student ... 34

5. Fifth Student ... 34

6. Sixth Student ... 34

7. Seventh Student ... 35

8. The Teacher ... 35

(16)

xiii

D. Instruments and Data Gathering Technique ... 36

E. Data Analysis Technique ... 37

F. Research Procedure ... 39

CHAPTER IV. RESEARCH FINDINGS AND DISCUSSION A. Justification of Epistemology ... 42

B. Washback Happen to the Parties ... 44

1. Students ... 45

2. Teachers ... 46

3. The School ... 50

C. Areas Affected by Washback ... 51

1. Curriculum ... 51

2. Teaching Materials ... 52

3. Teaching Methods ... 53

4. Feelings and Attitudes ... 55

a. Students’ Feelings and Attitudes ... 55

b. Teachers’ Feelings and Attitudes ... 56

c. The school’s Feelings and Attitudes ... 57

5. Learning ... 58

D. How the Washback Happened ... 59

1. The School ... 60

2. Teachers ... 60

3. Students ... 61

CHAPTER V: CONCLUSIONS AND RECOMMENDATIONS A. Conclusions ... 62

B. Reflections ... 63

B. Recommendations ... 64

1. For Teachers ... 64

(17)

xiv

3. For Future Researchers ... 65

REFERENCES ... 66 APPENDICES ... 73

(18)

xvi

LIST OF TABLES

Table page

Table 2.1 The Tracheotomy of Backwash Model ... 20 Table 3.1 The Coding ... 40 Table 4.1 Students’ Dependence on Exercises ... 52

(19)

xvii

LIST OF FIGURES

Figure page

Figure 2.1 Basic Model of Washback ... 21 Figure 4.1 Mechanism of Washback ... 59

(20)

xviii

LIST OF APPENDICES

Appendix page

(21)

1 CHAPTER I INTRODUCTION

This chapter consists of six subchapters. They are research background, research problems, problem limitation, research objectives, research benefits, and definition of terms.

A. Research Background

Indonesia has administered National Examination (NE) since 2005 whose results are used as one of the considerations to decide students’ graduation from secondary schools. Due to the deterministic nature of NE, it brings positive and negative impacts. On one hand, NE can motivate students to work harder as a response to achieve good result (as cited in news.okezone.com, April, 2013). On the other hand, NE can also create unnecessary anxiety and injustice among students, even teachers and schools (the parties: all that are involved). It is as expressed by an article entitled Fear of Failing in NE, A Girl Hangs Herself (Takut Tak Lulus UN, Seorang Siswa Gantung Diri) (megapolitan.kompas.com, May, 2013). Consequently, the implementation of NE also brings heated controversies and debates. Those are in line with Alderson and Wall (1993) who state that tests are conducted to be powerful determiners of what happens in classroom. The way in which NE affects teaching and learning is commonly known as washback or

backwash. To fill up the complete understanding about washback, a number of definitions from published research and literatures have been proposed.

(22)

Tests that have important consequences will have washback (Alderson & Wall, 1993). The important consequences here refer to a test in high-stakes situation. High-stakes tests themselves refer to the tests whose results are seen– rightly or wrongly–by the students, teachers, administrators, parents, or general public as being used to make important decisions that immediately and directly affect them (Madaus, 1988). In the other words, tests will have greater influence on teaching and learning in high-stakes situations than if it is not in high-stakes situations. Greaney and Kellaghan (1996) state that “an examination or test is said to have high-stakes attached to it when sanctions or rewards are linked directly to performance” (p. 37). Meanwhile, McNamara (2000) defines “a high-stakes test as a test which provides information on the basis of which significant decision are made about candidate” (p. 48).

There is some convincing evidence to suggest that examinations, especially high-stakes tests, have powerful washback effects on teaching and learning (Cheng & Watanabe, 2004). Smith emphasizes that the pressure to improve students’ test scores make some teachers neglect material that examination does not include and that teachers have negative feelings about standardized testing and narrowing of the curriculum (as cited in Cheng & Watanabe, 2004). High-stakes tests mostly result in the phenomena of teaching the test and teaching to test. It has the idea of what test questions will be and teachers teach the answers to the students. Noble and Smith (1944a) points that this method is likely to boost the scores but unlikely to promote the learning. It is only downright dishonest and can be considered as cheating.

(23)

Bachman and Palmer (1996) and Messick (1996) state that washback can be positive or negative, even neutral. Whatever the washback is, it always contains intended and unintended consequences (Qi, 2005). Meanwhile, the amount and type of washback depend on the extent to which: the status of the test, the degree to which the test counters to the current teaching practices, what teachers and textbook writers are willing and able to innovate (Alderson & Hamp-Lyons as cited in Cheng & Watanabe, 2004). High-stakes tests commonly appear in tests with important decisions such as admissions, promotion, placement, or passing out decisions that directly depend on the test scores. Meanwhile, for schools themselves, high-stakes decisions may include potential detachment of certain title or cuts in funding that may be directly dependent on score averages. Thus, the washback in high-stakes situations obviously give strong impacts to the parties.

In Indonesia itself, NE is still identical with fraudulent practices. Even though the government has done some steps to fight the fraudulences, it appears those fraudulences are still found in many regions in Indonesia. Fachrudin, the vice chief of Federation of Indonesian Teachers Alliance (FSGI), states that the current fraudulence level is almost the same with the previous year (republika.co.id, April, 2014). However, new modus operandi are found to be used by the subjects of fraudulences. Meanwhile, the center of applied psychology Universitas Pendidikan Indonesia (UPI), based on the online survey upon the NE from 2004 to 2013, found that NE fraudulences happened massively through cheating that involve teachers, principals, and supervisors (sp.berisatu.com, October, 2013).

(24)

Alderson and Wall (1993) say that the study of a washback phenomenon is always interesting. It is because washback is a complex phenomenon. Besides, they assume that teachers and learners do things they will not necessarily otherwise do because of the test. Furthermore, NE is still controversial in Indonesia. On the one hand, people who agree NE base their argument on the importance of NE as national quality control for education and motivation for educators and students to increase their quality. On the other hand, the chief of 10th commission of the parliament, Heri Ahmadi, states that NE causes school, which is a place to study, to lose its value since schools only focus on graduating their students (as cited in Pikiran Rakyat, December, 2007). Thus, this study seeks to display empirical evidence upon which NE brings either negative or positive washback effects, in particular, concerning a group of students, an English teacher, and a vice principal of an inner-city vocational high school in Yogyakarta.

A private religion-based vocational school in this research is a Christian-based private school that is under the authority of one of Christian-Christian-based foundation in Indonesia. This school is located in city of Yogyakarta. It happens to the context-bound of this study. Meanwhile, the participants also came from that school. The participants consist of students, teachers, and school management. For the consideration of ethical conduct, the researcher must not mention the name of the participants, the school, and the foundation.

Understanding some concepts of washback is required in this study. Deep understanding of washback concepts are used to understand the context of washback in the field of study. Besides, some phenomena following the washback

(25)

will also be reviewed to identify what kind of washback phenomena happen in the field of study. The researcher will also review some current models of teaching and learning in the context of theoretical and practical considerations of washback to identity how washback affect the teaching and learning in classroom.

B. Research Problems

This study has two research problems:

1. What washback do the twelfth graders of a private religion-based Vocational High School in Yogyakarta experience in response to National Examination preparation?

2. How does the washback happens to the students, teachers, and the school?

C. Problem Limitation

The limitation of this study is to discover the answers of two research problems. The first research problem is the washback experienced by the twelfth graders of a private religion-based Vocational High School in Yogyakarta in response to National Examination preparation. The second research problem is how the washback happen to the students, teachers, and the school. It is important to highlight that this study only focuses on the washback effects that happen during the preparation of NE. The scope of time will be from the beginning of the twelfth grade to the last day before NE.

(26)

The study will only focus on four aspects. Those are the washback will occur, areas affected by washback, response toward washback, and mechanism of how the washback happens.

D. Research Objectives

The objectives of this study are:

1. To find what washback experienced by the twelfth graders of a private religion-based Vocational High School in Yogyakarta in response to National Examination preparation.

2. To find how the washback happens to the students, teachers, and the school.

E. Research Benefits

This study gives benefits for students, teachers/English teachers, schools, and future researchers.

1. For students

This study is beneficial for students to give pictures to students of what they will face in preparing NE. Students will be able to anticipate the potential problems through the problems identified in this study. Furthermore, the suggestion presented in this study will help them more prepared for their NE in the future.

2. For Teachers/English Teachers

This study will help the English teachers in particular and teachers in general to understand more about the phenomenon faced by their students during the NE

(27)

preparation. Therefore, they can formulate some scenarios to give best anticipation for their students to reduce impacts from tests.

3. For Schools

School management, that acts as the policy makers in school, will understand more what phenomenon faced by their students relating to NE preparation. Hence, school management is expected to be more humble to students and even teachers rather than only standing in the administrative stages. Thus, schools can be closer to their students and together reducing the impacts of tests. 4. For Future Researchers

This study is also beneficial for future researchers who are interested in a washback phenomenon. Since the washback phenomenon is so complex, the washback phenomenon still provides many valuable issues to be studied. Thus, this study may inspire them in conducting the other research related to the washback phenomenon.

F. Definition of Terms

1. Washback

Hughes (1989) points out that washback is the effect of testing on teaching and learning process. Spolsky (1994) asserts that washback deals with the side effects of examinations. Meanwhile, Alderson and Wall (1993) say that washback compels teachers and learners to do things they will not necessarily otherwise do because of the test. Based on the definitions above, it can be accepted that washback is the side effects of testing.

(28)

2. National Examination

According to the Regulation of National Agency of Education Standard number 0022/P/BNSP/XI/2013, National Examination of SMP/MTs, SMPLB, SMA/MA, SMALB, SMK/MAK that is also known as NE is a measurement and assessment activity of achievement toward graduate competence standard of SMP/MTs, SMPLB, SMA/MA, SMALB, SMK/MAK nationally covered certain subjects.

3. Vocational High School

Evans states that vocational education is a part of education system that prepares its graduates to be able to work specifically on a certain field (as cited in Djojonegoro, 1997). Meanwhile, according to Law of National Education System number 20 year of 2003 article 15 stated that vocational education is a high school that prepares its graduates to work on a specific field.

4. Inductive Inquiry

Ary, Jacobs, and Sorensen (2010) state that inductive inquiry is a way in data analysis technique by using a particular set of facts to form a general idea. In this technique, researchers will usually categorize the data into many categories to ease composing conclusion. Categorization will ease researchers to conclude a particular set of facts.

5. Deductive Narrative

Deductive narrative is a way to present data by presenting the general idea in the beginning of paragraphs. According to Ary, et al. (2010), deductive narrative presents the general idea first and the supporting facts follow after. It is composed

(29)

based on the inductive inquiry done before. This technique will ease reader to understand the data. Thus, the discussion and analysis will give deep comprehension.

(30)

10 CHAPTER II

REVIEW OF RELATED LITERATURE

This chapter consists of two subchapters, namely the theoretical description and theoretical framework. Theoretical description presents the theories related to this study matter. Theoretical framework discusses all the major relevant theories to assist the researcher in answering the research problems.

A. Theoretical Description

Theoretical description gives a clear description of the theoretical underpinnings that shape and guide this study. It consists of three parts. The first part is exploring the concept of washback. The second part is exploring the research phenomenon. The third part is exploring the washback within vocational high school context.

1. Exploring the Concept of Washback

This part consists of three sections: the origin of washback, the definition and scope of washback, the washback defined in this study, and the function of washback.

a. The Origin of Washback

Experts have their various different terms to mention the influence of testing on teaching and learning. Washback (Alderson & Wall, 1993), and together with other similar related terms such as backwash (Biggs, 1996), test impact (Bachman & Palmer, 1996; Baker, 1991), systemic validity (Fredericksen & Collins, 1989),

(31)

consequential validity (Messick, 1996), measurement-driven instruction (Popham, 1983), curriculum alignment (Shepard, 1990, 1991), and possibly other terms, all refer to different aspects of the same phenomenon–the influence of testing on teaching and learning (Cheng, 2005). The researcher has chosen the term washback as the most commonly used in applied linguistics and references.

Alderson (1986) notices an additional area to which language testing research needed to turn its attention in years to come called the washback effect. He discusses the potentially powerful influence offsets. Washback is commonly introduced in testing for teachers as a powerful concept of which most classroom teachers are all too aware. Morrow (1986) uses the term washback validity for describing the quality of the relationship between testing and teaching and learning. He continues saying that conducting a study in washback validity of an examination is interesting for researchers.

Messick (1996) has put the washback effect within a broader concept, namely construct validity (consequential validity). He claims that construct validity covers facets of test use, the impact of tests on test takers and teachers, the interpretation of scores by decision makers, and the misuses, abuses, and unintended uses of tests. Washback is an inherent quality of any kind of assessment, especially when people’s future are affected by the examination result, regardless of the quality of the examination (Eckstein & Noah, 1992, 1993a, 1993b). Whether the washback effect is negative or positive can be so dependent on the examination itself. In short, washback is an educational phenomenon derived from research

(32)

studies into the relationship between teaching, learning, and testing. This is highly related to this study.

b. The Definition and Scope of Washback

Washback is a term commonly used in language testing, yet it is seldom found in dictionaries. However, the term backwash can be found in certain dictionaries. New Webster’s Comprehensive Dictionary of the English Language defines it as “the unwelcome repercussionsof some social action”. Meanwhile, the Collin’s Cobuild Dictionary of English Languagedefines it as the “unpleasant after -effects of an event or situation”.

Washback, commonly used in the field of applied linguistics, refers to the impact of a test on teaching (Alderson & Wall, 1993). It refers to the extent to which a test influences language teachers and learners to do things they will not necessarily otherwise do because of the test (Alderson & Wall, 1993). Messick (1996) notices that washback, a concept prominent in applied linguistics, refers to the extent to which the introduction and the use of a test influences language teachers and learners to do thing they will not otherwise do that promote or inhabit language learning. He continues to comment that some proponents have even maintained that a test’s validity should be appraised by the degree to which if it manifests positive or negative washback, a notion to akin to the proposal of “system validity” (Frederiksen & Collins, 1989) in the education measurement literature. Shohamy (1992) notes that this phenomenon is the result of the strong authority of external testing and the major impact it has on the lives of test takers.

(33)

Biggs (1995) uses the term backwash to refer to the fact that testing drives not only the curriculum, but also teaching methods and students’ approaches to learning (Crooks, 1988; Frederiksen, 1984; Frederiksen & Collins, 1989). However, after reviewing the definitions of the term backwash from the Collin’s

Cobuild Dictionary of English Language, Spolsky (1994) comments that backwash is better applied only to accidental side effects of examinations, and not to those effects intended when the first purpose of the examination is control of curriculum. According to Alderson and Wall (1993), the notion that testing influences teaching is referred to as backwash in general educational circles, but it has come to be known as washback among British applied linguistics, though they see no reason, semantic or pragmatic, for preferring either term.

Pearson (1988) notices that public examinations influence the attitude, behaviours, and motivation of teachers, learners and parents, and because examinations often come at the end of a course, this influence is seen working in a backward direction, hence the term washback. He further emphasizes that the direction in which washback actually works must be forwards in time. Alderson and Wall (1993) also emphasize the fact that evidence of washback is typically demonstrated in behavioral and attitudinal changes in teachers and learners that are associated with the introduction of tests bearing important educational consequences. Meanwhile, Spratt (2005) points out that washback effects on various aspects of classroom, which can be categorized as curriculum, teaching materials, teaching methods, feelings and attitudes, and learning. In relation to curriculum, Alderson and Wall (1993), from their Sri Lanka study, note that the

(34)

examination has had a demonstrable effect on the content of language learning. This effect is the narrowing down the curriculum to those areas most likely to be tested. It is similar to what Lam (1994) notes that he reports an emphasis in teaching on those parts of exam carrying the most marks. In the teaching materials, Alderson and Wall (1993) point out the rising use of past papers and commercial exam-public publications, as the exam is getting closer. Teaching methods refer to the teaching approaches and techniques. Shohamy (1996) states that teaching toward high-stakes EFL exam lead the teachers to teach through simulating the exam tasks or carrying out other activities directly aiming at the developing exam skills or strategies. Engaging with feelings and attitudes, Cheng (1998b) mentions that students show mixed feelings towards the examination itself, recognizing on one hand that examination makes students work to achieve good scores but at the same time thinking that examination are not an accurate reflection of all aspect their study. The examination itself has the potential to generate an atmosphere of high anxiety and fear of test results among teachers and students (Spratt, 2005). Alderson and Hamp-Lyons (1996) say that most of the teachers have negative attitude towards the examination and teaching. They also mention that teachers’ feelings of guilt and frustration will also make them to be unable to make the content interesting and ensure improved scores of their students. In relation to learning, the washback engages with the question whether washback affect learning, and if so, how? Wall (2000) says what is missing are analyses of the results which indicate whether students have learned more or better because they have studied for particular test.

(35)

Messick (1996) mentions that for promoting positive washback, teacher should involve different activities between in regular learning and preparing for the test. Alderson and Wall (1993) argue that washback is likely to be a complex phenomenon that cannot be related directly to a test’s validity. The washback should refer to the test itself on aspects of teaching and learning. Besides, other operating forces within the education context also contribute to or ensure that washback effect takes its place on teaching and learning.

Bailey (1996) summarizes, after considering several definitions of washback, that washback is defined as the influence of testing on teaching and learning, that it is widely exist and to be important to pay attention at. He comments further that there are also some concerns about what constitute both positive and negative washback, as well as about how to promote the former and inhabit the latter.

c. Washback Defined in this Study

The term washback, after reviewing some definitions of washback, can be defined into two major perspectives, i.e. the narrower view or washback within the classroom (micro level), and the broader view beyond the classroom (macro level). Bachman and Palmer (1996) suggest that washback at macro level refers to the extent to which a test influences within the society, ranging from the government policy making, school administration, publishing, and general opportunities, to parents’ expectation of their children. Meanwhile, at micro level, washback refers to the extent to which a test influences within the classroom, mainly in the change or innovation of curricula and teachers’ methodologies and

(36)

the influence of students’ learning. Bailey (1996) uses the phrases “washback to the learners” to indicate the effects of test on students, and “washback to the programme” to indicate effects of the test on teachers, administrators, curriculum developers, counselors, etc.

In short, the narrower definition washback focuses on the effect that a test has on teaching and learning. The broader definition looks beyond the classroom to the educational systems and society. Thus, tests can have “significant impact not only on individuals, but also on practices and polices–in the classroom, the school, the educational system, and in the society (Wall, 2005).

In this study, the researcher prefers to use the term washback to describe the narrower interpretation of washback to investigate the impact of NE on teaching and learning, and also by looking at some possible factors of macro level of washback that may involve.

1) Negative Washback

Language tests are often to get many criticisms for their negative influence on teaching–so-called negative washback (Alderson & Wall, 1993). Shohamy (1992) notes that test will lead to the narrowing down of curriculum. Students tend to learn only to pass the test rather than total phases of understanding. Teachers tend to ignore the subjects and activities that are not directly related to passing the examination, and tests accordingly alter the curriculum negatively (Vernon, 1956). Davies (1968), for instance, indicates that it is too often that washback effect has been bad: designed as testing devices, examinations have become teaching devices, work is directed toward what are in past examination papers and consequently

(37)

becomes narrow and uninspired. Shoahamy (1996) also points out that many teachers detail high anxiety, fear, and pressure to cover the material, as they feel that their job performance is assessed by the students’ test scores. Alderson and Wall (1993) refer to the negative washback as the negative or undesirable effect on teaching and learning of a particular test.

Noble and Smith (1994a) point out that high-stakes testing affects teachers directly and negatively, and that “teaching test-taking skills and drilling on multiple-choice worksheets is likely to boost the scores but unlikely to promote general understanding” (1994b). Smith (1991) concludes that testing programs substantially reduce the time available for instruction, narrow curricular offerings and modes of instruction, and teach content and to use methods and materials that are incompatible with standardized testing formats.

2) Positive Washback

Some researcher believe that it is feasible and desirable to bring about beneficial change in language teaching by changing examinations, which refer to so-called positive washback. Positive washback refers to tests and examinations that influence teaching and learning beneficially (Alderson & Wall, 1993). In this sense, teachers and learners have a positive attitude towards the test and work willingly toward its objectives. Alderson and Wall (1993) further explains that teachers and learners will be motivated to fulfill their teaching and learning goals. Pearson (1988) argues that good test will be more or less directly usable as teaching-learning activities. In the same way, good teaching-teaching-learning tasks will be more or

(38)

less directly usable for testing purposes, even though practical or financial constrains limit the possibilities.

Shohamy (1992), Alderson and Wall (1993), and Cheng (2005) maintain the view that decisional makers use the authority power of high-stakes testing to achieve the goals of teaching and learning, such as the introduction of the new textbooks and curricula. Heyneman (1987) concludes that testing is a profession, but it is highly susceptible to political interference. In the large extent, the quality of tests rely on the ability of a test agency to pursue the professional ends autonomously. If the consequences of the teaching and learning are to be evaluated, the educational context in which the test takes place need to be investigated. Thus, whether the washback is positive or negative will largely depend on how it works within which educational contexts it is situated (Cheng, 2005).

d. The Function of Washback

Tests, traditionally, come at the end of the teaching and learning process for the purposes of evaluation (Cheng & Watanabe, 2004). However, with the widespread of high-stakes public examination systems, they further explain that the direction can possibly change. They also explain that, particularly, when tests are used as levers for change, new materials need to be designed to match the purposes of a new test, and school administrative and management staff, teachers, and students are generally required to learn in alternative ways, and often work harder, to achieve high scores on the test. In addition, many more changes in the teaching and learning context can occur as the result of the test, although the consequences

(39)

and the effects may be independent towards the test takers, due to the complex interplay of forces and factors both within and beyond the school.

Such influences are linked to test validity by Shohamy (1993a), who points out that the need to include aspects of test use in construct validation originates in the fact that testing is not an isolated event; rather, it is a connected to a whole set of variables that interact in the educational process. In the same way, Linn (1992) encourages the measurement research examination system should pay greater attention to investigations of both the intended and unintended consequences of the system than was typical of previous test-based reform efforts.

The complexity motivates Messick (1989) to recommend a unified validity concept, in which he shows that when an assessment model is designed to make inferences about a certain construct, the interferences drown from that model should not only derive from test score interpretation but also from other variables in the social context (Messick, 1992). Messick (1975) points out that researchers, other educators, and policy makers must work simultaneously to develop means of evaluating educational effectiveness that accurately represent a school or district’s progress towards a broad range of important educational goals. Relating to this context, Linn (1992) emphasizes the important to pay greater attention to investigations of both the intended and unintended consequences toward any high-stakes examination system.

Hughes (as cited in Bailey, 1996) suggests a framework in order to clarify thinking about backwash. He explains that the framework will be helpful to distinguish between participants, process, and product in teaching and learning, and

(40)

to recognize those three may be affected by the nature. In the framework, participants include language learners and teachers, administrators, material developers, and publishers, all of whose perceptions and attitudes toward their work may be affected by the test. The term process encompasses any actions taken by the participants that may contribute to process of learning. Hughes says that such processes include materials development, syllabus design, changes in teaching methods or content, learning and/or test-taking strategies, etc. Meanwhile, the last term, product, refers to what it learned (facts, skills, etc.) and the quality of learning (fluency, etc.). He continues to use tracheotomy to illustrate the mechanisms how washback occurs in actual teaching and learning environments (see Table 2.1). He explains his model as follows:

The tracheotomy … allows us to construct a basic model of backwash. The nature

of a test may first affect the perceptions and attitudes of the participants towards their teaching and learning tasks. These perceptions and attitudes in turn may affect what the participants do in carrying out their works (process), including practicing learning outcomes, the product of the work.

Table 2.1 The Tracheotomy of Backwash Model

(a) Participants–students, classroom teachers, administrators, materials developers and publishers, whose perceptions and attitudes toward their work may be affected by test

(b) Processes–any action taken by the participates which may contribute to the process of learning

(as cited by Hughes in Bailey, 1996)

The framework suggested by Hughes (1993) that consider the participants, processes, and products in examining washback lead Bailey (1996a, p. 264) to draft the following model:

(41)

(Bailey, 1996a, p.264)

Figure 2.1 Basic Model of Washback

Alderson and Wall (1993), on the other hand, focus on micro aspects of teaching and learning that may be influenced by examinations in their Sri Lanka study. They propose 15 hypothesis regarding washback (1993, p. 120 – 121), to illustrate areas of teaching and learning affected by washback. The hypothesis are as follows:

1. A test will influence teaching. 2. A test will influence learning.

3. A test will influence what teachers teach; and 4. A teacher will influence how teachers teach 5. A test will influence what learners learn

PRODUCTS PROCESSES

PARTICIPANTS

TEST

Students

Teachers

Material writers and curriculum designers

Researchers

Learning

Teaching

New materials and new curricula

Research results

(42)

6. A test will influence how learners learn

7. A test will influence the rate and sequence of teaching; and 8. A test will influence the rate and sequence of learning 9. A test will influence the degree and depth of teaching 10. A test will influence the degree and depth of learning

11. A test will influence attitude towards the content, method, etc. of teaching and learning

12. Tests that have important consequences will have washback; conversely, 13. Tests that do not have important consequences will have no washback 14. Tests will have washback on all learners and teachers

15. Tests will have washback effects for some learners and some teachers, but not for others.

Wall (1996), following the above study, stresses the difficulties in finding explanation of how tests maintain influence on teaching. Alderson and Wall (1993) concludes that further research on washback is needed, and the research must specify the above washback hypothesis more.

2. Exploring the Research Phenomenon

This section consists of two parts, namely high-stakes testing and measurement-driven instruction.

a. Influences of High-stakes Testing

High-stakes testing refers to tests whose results are seen–rightly or wrongly–by the students, administrators, parents, or the general public as the basis upon which important decisions are made that immediately and directly affect the student (Madaus, 1988). Cheng (2004) states high-stakes tests offer academic and employment opportunities based on their results. They are usually public examination and standardized tests.

According to the Regulation of the Educational and Cultural Ministry number 3 year of 2013, article 2d, passing out the NE becomes one of the requirement to graduate from secondary school. NE pure score itself is worth 40%

(43)

that later will be combined with 60% from the school score to get the final decision of graduation. Meanwhile, NE also functions as a basis selection towards the above level of education (university). In addition, NE in past two years has administered 20 test packets (NE Socialization for VHS from Educational, Youngster, and Sport of Yogyakarta, 2013, 2014). It means each person in the examination room has different test packet. Thus, the subject of the study, the NE in a religion-based private Vocational High School in Yogyakarta, is such a high-stakes test.

Cheng (2004) finds quite a number of studies, both general in nature and in language education, illustrating that high-stakes standardized testing influences teaching and learning. The major areas that have been studied are as follows:

 School (organization, practices, and achievement; accountability pressure)

 Teachers (their attitudes and responses to standardized tests; teachers’

attention to testing in instructional planning and delivery; teaching contents; time spent on test preparation; teachers’ sense of professional

pride; and teachers’ general attitudes about the fairness and utility of

testing, etc.)

 Students (their reactions, self-concepts, and self-assessment; and student learning outcomes)

 Parents (familiarity with the changes in evaluation of their children’s school progress, and their knowledge about attitudes towards standardized testing).

Many studies have shown that high-stakes testing influences teaching and learning in the following ways. In 1990, Smith, Edelsky, Draper, Rottenberg, and Cherland find that pressure to improve students’ test scores encourages some teachers to neglect the material that the external test does not include. In the other study, Smith (1991b) comments that teachers have negative feelings about standardized testing and the narrowing down the curriculum. Cheng says that teachers alter their instructional materials to resemble the format of the standardized tests (as cites in Mathison, 1987). Teachers also tend to look at prior tests to make

(44)

sure that their instruction includes all or most of the test content, and plant to ensure that they cover all test objectives (Herman & Golan, 1991). Furthermore, Madaus (1988) notes that teachers teach to the test when they believe important decisions will be based on the test scores. In conclusion, it appears that the grater the consequences brought by the test, the more likely it will be to have an impact on teaching (Alderson & Wall, 1993).

b. Measurement-driven Instruction

Cheng says examinations as an encroaching power that is influencing education, blurring, distinctions between liberal and technical education, and narrowing the range of learning through forcing students to prepare by studying with crammers and in cramming schools (as cited in Lathan, 1877). Spolsky (1994) emphasizes that the encroaching power also permits any external body so-called a university or a government agency to exert control over the internal operations of educational systems that are becoming increasingly complex.

Popham (1987) says that MDI happens when a high-stakes standardized test influences a teacher’s instructional program in order to prepare students for the test. He also outlines the traditional notion of MDI to illustrate the relationship between instruction and assessment. He suggests that assessment directs teachers’ attention to the content of test items, acting as powerful “curricular magnets”. Furthermore, he states that, in high-stakes situation, in which the results of standardizes tests trigger rewards, sanctions, or public scrutiny and loss of professional status, teachers will be motivated to pursue the objectives that the test embodies. Noble and Smith (1994b) points out that the current MDI reform in assessment aims to

(45)

build better test that will drive school towards more ambitious goals. It, in turn, will drive a shift from behaviorism to cognitive-constructivism (a shift to a curriculum and pedagogy run more towards thinking instead of rote memory and isolated skills) in teaching and learning beliefs.

Cheng (2005) states that beliefs about testing reflect beliefs bout teaching and learning. Whatever the objectives covers, tests can be written to measure them, can be administered to all, and re-administered as necessary to those who fail (Noble & Smith, as cited in Cheng, 2004, p. 36).

Cheng (2005) states that behaviorist psychology and pedagogy underlies the traditional teaching and learning views from which views of traditional testing are derived. The view says that the desired performance of students is brought by reinforcing successive approximations of correct performance. Students are presented with the academic tasks that are broken down into discrete units. Correct performance bases rewards and progress through an ordered hierarchy of tasks and skills. Inadequate responses from high order skills instruction used will cause repetition and repeating through the same material until it is mastered. In behaviorist model, students are considered passive recipient of knowledge. Teachers ignore their intentions.

On the contrary, the more recent psychological and pedagogical view, labelled cognitive-constructivism, emphasize three interrelated dimensions.

1. Learning is viewed as a process of construction. In this view learning is not an act of recording discrete pieces of information, each piece independent of the others and needing only to be repeated until it is mastered. It is a process of interpretation and construction of meaning (Glaser & Bassok, 1989). Students are active participants in their learning, constructors of knowledge, not passive recipients of information and skills (Piaget, 1973).

(46)

2. Learning is knowledge-dependent. It is not merely an act of receiving, but one of interpreting information through earlier learning. the role of prior knowledge and experience is given special attention, for what a student learns on a given occasion is dependent on what has been already learned. It is much easier for individuals to learn more areas of their expertise than it is for them to learn about other topics outside their experiences.

3. Learning is situated in a context (Biggs, 1992, 1993, 1995; Resnick, 1989; Resnick & Resnick, 1992). Knowledge is not independent of the context in which it develops. Learning not only occurs in a context but also is social. Learning cannot be separated from the context in which it occurs (Gergen, 1995).

(Cheng, 2005, p. 36-37)

The perspective above proposes that effective instruction must connect with how students think. Cheng (2005) says that the direct instruction from behaviorism does not match with how students learn. Cognitive-constructivist teaching and learning view is likely to be more holistic, integrated, long-term, discovery-based, and social. Hence, Cheng further states that cognitive-constructivism see performance assessment or testing as parallel to the above belief of how students learn and how they should be taught. Gipps (1994) states that teachers will use previously acquired knowledge to measure learners’ ability in solving novel problems or completing specifics tasks. In performance assessment, a qualified judge uses real life or simulated assessment exercises to elicit original responses that are directly observed and rated.

Furthermore, Cheng (2005) states that the proponents of performance assessment believe that what is assess is what is taught. In the same sense of this belief, Resnick and Resnick (1992, p. 59) states that (1) you get what you assess, (2) you do not get what you do not assess, and (3) you should build assessments towards how you wants educators to teach. Cheng (2005) says that high-stakes

(47)

testing can drive reform if it follows better psychology and pedagogy and employ more appropriate measurement forms–the performance assessments.

In addition, one important assumption related to high-stakes testing (or public examination), as mentioned before), is that the deleterious impacts associated with performance-based assessments closely linked to curriculum framework. The argument is instruction directed towards preparation for a performance-based assessment promotes better instructional practice (Cheng, 2005). She argues that better test will produce better results. Performance-based assessment can be designed to be closely linked to the goals of instruction as to be almost indistinguishable from them. In spite of being a negative consequence, as it is now with some high-stakes uses of existing standardized tets, teching to these proposed performance assessment, accepted by schoalars as inevitable and teachers as necessary, become a virtue, according to this line of thinking (Noble & Smith, 1994b).

Cheng (2004) concludes that measurement can drive instruction in positive a positive way. However, it will happen in a condition if measurement is properly conceptualized, or in the above namely performance-based assessment.

3. Washback within Vocational High School Context

Evans (as cited in Djojonegoro, 1997) states that vocational education is a part of education system that prepares its graduates are able to work specifically on a certain field. Another definition says that vocational education is an educational program that directly prepares individuals to a certain job expertise. Meanwhile, according to Law of National Education System number 20 year of 2003 article 15

(48)

stated that vocational education is a high school that prepares its graduates to work in a specific field. In short, it appears that vocational education prepares their graduates directly for the working world.

Kumaat (2010) points out a perception spreading among the society that VHS is the “second class” compared to the senior high school (SHS). He continues that it is because the government designs VHS for the graduates to be ready directly for the working world. The perception becomes so reasonable because the amount of parents who send their children to VHS came from the middle to low society, based on his observation in North Sulawesi. It is so logical then, and if their children cannot extent their education to colleges, their children have already had the ability to work or even be an entrepreneur. It is what happens, since individuals who go to SHS to be certain want to go to colleges.

This study focuses on studying the washback of National Examination preparations in that school.

B. Theoretical Framework

The theoretical framework summarizes and synthesizes the relevant theories explained above. In this section, the researcher relates the study matter to the theories.

To answer the research problems, the researcher implements the theories that are appropriate for the topic of this study. The first section of the theoretical description focuses on the nature of washback and how washback works in education. The concept of washback is first constructed by the origin and the

(49)

definition of washback. Then, the scope of washback illustrate the tremendous impact and power of testing on teaching and learning in school. It focuses mainly on the micro level of washback, but it also still opened with the macro level of washback. Meanwhile, the washback defined in this study will give picture of the focus of washback effect studied in this study. Cheng (2005) states that washback effect, whether it is negative or positive, is still debatable and needs to be studied further. Furthermore, negative and positive washback also provide notions on what situation it can be whether negative or positive. At last, the function of washback functions to find the meaning and the mechanism of how washback works in teaching and learning through the framework proposed by Hughes (as cited in Bailey, 1996), namely the Hughes’ trichotomy.

The second section of chapter consists of two interconnected parts, namely influences of high-stakes testing and measurement-driven instruction. It explores how and why washback can work to influence other components in educational system, trace the rationale the use of NE, and its powerful function to change teaching and learning.

The third section of this chapter is used to put washback phenomenon studied under the microscope and into the specific area of educational context. It presents the nature of the vocational education and the issues commonly follows. It will be used later to construct the logic and rationale to examine the washback effect in this study.

(50)

30 CHAPTER III

RESEARCH METHODOLOGY

This chapter presents the methodology used in this study. This chapter consists of six subchapters, namely research method, research setting, research participants, instrument and data gathering technique, data analysis technique, and research procedure.

A. Research Method

The research method used in this study was qualitative research. Lichtman (2013) defines qualitative research as a way for a researcher to gather, organize, and interpret information obtained from humans by using their eyes and ears as filters. Ary, Jacobs, and Sorensen (2010) state that by using qualitative research, a researcher is able to look for the understanding of the phenomenon by emphasizing on the holistic picture. It does not rely heavily on hypothesis testing, cause and effect, and statistical data. Lichtman (2013) says that the researcher plays important role in qualitative research. A qualitative researcher uses his or her eyes and ears to collect and gather the data and information. Ary, et al. (2010) say that qualitative research aims to get a full and in-depth understanding of a picture. Hence, it concerned with the question “why” to analyze the phenomena instead of relying on measuring and counting. In addition, Lichtman (2013) points out that the goal of qualitative research is to understand and interpret social interactions. Furthermore, qualitative research will assist the researcher to investigate what washback

(51)

phenomenon occurs and how it can occur. Thus, the researcher used qualitative research in this study.

This study used case study as the method. Ary et al. (2010) state that a case study focuses on a single unit to produce a rich and holistic in-depth description. The “unit” mentioned before could be a group, an individual, a class, a site, a program, a policy, a process, a community, or an institution. The unit should be defined within a specific bounded system for its uniqueness or special characteristic. The unit focused on this study was the washback phenomenon of a religion-based VHS in the preparation for NE. The researcher considered case study can be the best method used to answer the research questions. Ary et al. (2010) point out that case study gives possibility of depth. It seeks to understand the holistic case in the environment. It was not limited only the present actions, but also possible to the past, emotions, thoughts, and environment. Thus, the holistic data would be optimally obtained through case study.

B. Research Setting

This study was conducted in a private religion-based VHS in Yogyakarta. Due to the consideration of ethical conduct, the researcher must not mention the school. The setting of time in this research was divided into two phases. The first phase was used for observation. It occurred during the researcher had the Program Pengalaman Lapangan (PPL) from July to December 2013. The second phase was data gathering. It occurred from March to April 2014.

(52)

C. Research Participants

The researcher relied on focus groups to obtain the data. Ary, et al. (2010) state that focus groups rely on purposive sampling, just like most quantitative research did. They further explain that a purposive sampling strategy chooses the focus group’s participants according to the project’s goal.

Purposive sampling is very different from the random sampling that is so common in survey samples. Ary, et al. (2010) state that surveys aim to generelaize the larger populations by collecting numerically precise data, and it requires selecting a random sample to cover the entire range of the larger population. Meanwhile, the focus groups aim to gain insight and in-depth understanding by hearing from people, and it requires selecting a purposive sample to generate the most productive discussions in the focus groups.

The participants in this research consisted of three elements–student element, teacher element, and school element. Student element consisted of seven students that had been classified into three classifications–high, medium, and low. The classification was based on their academic capability. A twelfth grade teacher suggested the classification. The teacher acted as an advisor for the researcher in conducting the research. The teacher, later on, would be known as the advisor. Teacher element came from the English teacher who taught twelfth grade. School element was the vice principal from the curriculum subdivision.

1. First Student

The first student belonged to the high classification. The first impression when the researcher knew her was she was talkative, brave, and warm. She could

(53)

maintain enjoyable interaction even though with new people. It was no doubt that she was the head of Scouts organization in her school. Based on the advisor, she was also one of the brightest students in her class.

The first student came from a middle to low family. In her family, she was the first children. She still had little brothers and sisters. Fact as the oldest child gave her more responsibility in her family. In the morning, she had to help her mother to take care of her little sisters and brothers while her mother was busy in her family small shop. After school, she had to rush home to help her mother to take care of her little sisters and brothers so that her mother could focus on working in the small shop. In the evening, she usually went to market to take some goods for sale. She did all her family duty without leaving her duty at school. It is included when she was in the NE preparation. She readjusted her schedule in order to catch up all the agenda. It appeared that her life was tough. Even though it was tiring for her, she said that she did not have reason to complain. Instead, it built her to be strong.

2. Second Student

The second student came the high classification. He belonged to the same class with the first student. Based on the advisor, he was also one of the brightest students in his class. He was a bit shy at the first impression. However, after talking some words, he was actually talkative and warm as well.

He lived in an orphanage and belonged to low-economic family as well. He said that the entire fund he spent for his education came from the foundation of his orphanage. Besides, he also got scholarship from his school for his academic

(54)

achievement to subsidize his educational fund. Hence, he worked hard in preparing NE to get flying score since he dreamed a scholarship to go to a university. Unfortunately, he did not get any scholarship at last and now he worked as waitress. 3. Third Student

The third student was from the medium classification. She was a little bit indifferent at the first impression. It also appeared in her answer that she only answered in one to two words. The researcher needed to stimulate her with more questions to get information from her. She belonged to the middle to low family. She said that she wanted to work after graduating from VHS.

4. Fourth Student

The fourth student came from the medium classification. She was a little bit shy and indifferent. However, after throwing some words, she was more talkative than the third student. She also belonged to the middle to low family. She dreamed to get a job after graduating.

5. Fifth Student

The fifth student belonged to the medium classification. She was talkative and a little bit boyish. Instead, it eased the researcher to interview her since she was so opened. Besides, she was also independent. She lived alone in Yogyakarta. After graduating from VHS, she dreamed to go to a university.

6. Sixth Student

The sixth student belonged to the low classification. However, she came from a middle to high family. She was the only one among all participants who join NE preparation course. Besides, she also wanted to go to university. One interesting

(55)

side from her was that she often misunderstood with the question thrown. The researcher had to repeat the same question for sometimes to get her in line with the question.

7. Seventh Student

The last student belonged to the low classification. He did not dream to go to university since he know that he came from middle to low family. He wanted to work after graduating. Besides, he was actually an active student. He also joined Scouts organization in his school.

8. The English Teacher

She was the English teacher who taught twelfth grade. He also became an academic advisor for one of the majors in twelfth grade. Even though she was not a permanent teacher, she had experienced to prepare twelfth students four times. She was a talkative teacher and be able to blend with their student easily.

9. The Vice Principal

The researcher chose him as the representation of the school authority. As vice principal of curriculum subdivision, he knew exactly the strategic plan to face the NE. Besides, he also taught Math that is one of the NE subjects. The researcher hoped to obtain the information related to the policies taken by the school in response to the NE.

(56)

D. Instruments and Data Gathering Technique

The instruments used in this study were the researcher himself, observation, and interview. The researcher himself was going to conduct the research in this qualitative study. Ary, et al. (2010) state that human can be an instrument for a study. They further explain that human investigator is the primary instrument for gathering and analyzing data in qualitative studies. In addition, Lichtman (2013) states that researcher is the primary instrument since he or she who gathers and collect the data and information. Thus, the researcher himself would gather and analyze the data in this study.

The second instrument on the research was observation. Ary, et al. (2010, p.431) say “observation is a basis method for obtaining data in qualitative research and is more than just hanging out”. They further explained that observation used in

qualitative research is more global instead of systematic and structured observation. The observation aims to complete the description of behavior in specific setting. While conducting PPL in the school or later called as the field or research, the researcher could widely observe the behavior of school inhabitants in specific setting. He could even go to the classroom to observe what happened in classroom. Furthermore, the researcher could have direct communication to the school inhabitants to obtain some necessary detailed information.

The third instrument in this research was the interview. Interview is one of the most widely used and basic methods for obtaining qualitative data (Ary, et al., 2010, p.438). It is used to gather people-based data about their opinions, beliefs, and feelings about situation in their own words. Hence, it can focus on looking for

(57)

and making meaning (Lichtman, 2013). In conducting the interview, the researcher was assisted with the interview protocol. However, the researcher preferred to conduct semi-structured interview to maintain flowing communication between the researcher and the interviewees. Meanwhile, the type of questions used in the interview were open-ended questions. Ary, et al. (2010) note that it is useful to reveal what is important to understand under the study. In gathering the interview data, the researcher was assisted by an audio recorder device. Ary, et al. (2010) say using audio recorder is one of the most effective ways to collect interview data.

E. Data Analysis Technique

In qualitative research, there are three data analysis techniques–organizing and familiarizing data, coding and reducing, and interpreting and representing (Ary,

et al., 2010). Organizing and familiarizing data aimed to ease the retrieved data. Since the data in this study obtained from observation and interview, initially, the researcher collected the notes from the observation and made transcription from the interviews. Ary, et al. (2010) say that words should be transcribed as they exist to avoid bias. It was included the authentic grammar used by the interviewees, and the nonverbal information such as gestures and laughter. To become familiar with the data, the researcher should familiarize themselves with the data through reading and rereading the observation notes and transcript. Afterward, the researcher should make the complete list data sources and organize the data based on the list. Thus, the researcher would have familiarized with the organized observation notes and transcript in the first stage.

(1)

Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 97-111). Mahwah, NJ: Lawrence Erlbaum Associates.

Herman, J. L., and Golan, S. (1991). Effects of standardized testing on teachers and learning–Another look. CSE Technical Report 334. LA: University of California, CRESST.

Heyneman, S. P. (1987). Use of examinations in developing countries: Selection, research, and education sector management. International Journal of Education Development, 7, 251–263.

Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press.

Kellaghan, T. & Greaney, V. (1992). Using examinations to improve education: A study of fourteen African countries. Washington, DC: The World Bank. Kumaat, H. (2010). Persepsi masyarakat terhadap Sekolah Menengah Kejuruan

(SMK) sebagai upaya memasuki dunia kerja. APTEKINDO.

Lam, H.P. (1994). Methodology washback -- an insider's view. In Nunan D., Berry R., & Berry V. (Eds.), Bringing about change in language education: Proceedings of the International Language in Education Conference 1994 (pp. 83-102). Hong Kong: University of Hong Kong.

Linn, R. L. (1992). Educational assessment: Expanded expectations and challenges (Tech. Rep. 351). Boulder: University of Colorado at Boulder, Center for the Study of Evaluation.

Litchman, M. (2013). Qualitative research in education: A user’s guide (3rd _ed.). London: SAGE Publication, Inc.

Madaus, G.F. (1988). The influence of testing on the curriculum, in Tanner, L.N. (Ed.) critical issues in curriculum: Eighty-seventh yearbook of National Society for the Study of Education. Chicago: University of Chicago: University of Chicago Press, 83-121.

McNamara, T. (2000). Language testing. Oxford: Oxford University Press.

Merriam, S.B. (2009). Qualitative research: A guide to design and implementation. San Francisco: Jossey-Bass.

Messick, S. (1975). The standard problem: Meaning and values in measurement and evaluation. American Psychologist, 30, 955–966.

(2)

Messick, S. (1989). Validity. In R. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). New York: Macmillan

Messick, S. (1992, April). The interplay between evidence and consequences in the validation of performance assessments. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.

Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3):241-256.

Morrow, K. (1986). The evaluation of tests of communicative performance, in Portal, M. (Ed.) Innovations in language testing: Proceedings of the IUS/NFER Conference. London: NFER/Nelson, 1-13.

Noble, A. J., & Smith, M. L. (1994a). Measurement-driven reform: Research on policy, practice, repercussion (Tech. Rep. 381). Tempe, AZ: Arizona State University, Center for the Study of Evaluation.

Noble, A. J., & Smith, M. L. (1994b). Old and new beliefs about measurement-driven reform: ‘The more things change, the more they stay the same’ (Tech. Rep. 373). Tempe, AZ: Arizona State University, Center for the Study of Evaluation.

Pearson, I. (1988). Tests as levers for change, in Chamberlain, D., and Baumgardner, R. J. (Eds.) ESP in the classroom: Practice and evaluation. Great Britain: Modern English Publication, 98-107.

Popham, W.J. (1983). Measurement as an instructional catalyst, in Ekstrom, R.B. (Ed.), Measurement, technology, and indivuality in education: Proceedings of the 1982 ETS Invitational Conference, New Directions for Testing and Measurement, 17, San Francisco: Jossey-Bass, 19-30.

Qi, L. (2004). Has a high-stakes test produced the intended changes? In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 171-190). Mahwah, NJ: Lawrence Erlbaum Associates.

Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educational reform. In B. R. Gifford &M. C. O’Connor (Eds.), Changing assessments: Alternative views of aptitude, achievement and instruction (pp. 37–75). London: Kluwer Academic.

Shephard, L.A (1990). Inflated test score gains: Is the problem old norms or teaching the test? Educational Measurement: Issues and Practice 9, 15-22.

(3)

Shephard, L.A. (1991). Psychometricians’ belief about learning. Educational Researcher 20 (6), 2-16.

Shohamy, E. (1992). Beyond proficiency testing: A diagnostic feedback testing model for assessing foreign language learning. Modern Language Journal, 76:513-521.

Shohamy, E. (1993b). The exercise of power and control in the rhetorics of testing. Center for Applied Language Studies, Carleton University, Ottawa, Canada, 10:48–62.

Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test impact revisited: Washback effect over time. Language Testing, 13(3):298-317.

Smith, M.L., Ederly C., Draper K., Rottenberg C., and Cgerland M. (1990). The role of testing in elementary schools. CSE Technical Report 321. LA: University of California, CSE.

Smith, M. L. (1991a). Meanings of test preparation. American Educational Research Journal, 28, 521–542.

Smith, M. L. (1991b). Put to the test: The effects of external testing on teachers. Educational Researcher, 20(5), 8–11.

Spolsky, B. (1994). The examination of classroom backwash cycle: Some historical cases, in Nunan, D., Berry, V. R. (Eds) Bringing about change in language education. Hong Kong: University of hong Kong, Dept. Curriculum Studies, 55-66.

Spratt, M. (2005). Washback and the classroom: The implications for teaching and learning studies of washback from exam. Language Teaching Research, 9 (1), 5-29.

Vernon, P.E. (1956). The measurement of abilities. London: University of London Press.

Wall, D. (1996). Introducing new tests into traditional systems: Insights from general education and from innovation theory. Language Testing, 13, 334– 354.

Wall, D. (2000). The impact of high-stakes testing on teaching and learning: Can this be predicted or controlled? System, 28, 499–509.

Wall, D. (2005). The impact of high-stakes examinations on classroom teaching. Cambridge: Cambridge University Press.

(4)

Watanabe, Y. (2004). Methodology in washback studies. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 19-36). Mahwah, NJ: Lawrence Erlbaum Associates.

Watanabe, Y. (2004). Teacher factors mediating washback. In L. Cheng, Y. Watanabe, & A. Curtis, (Eds.), Washback in language testing: Research contexts and methods (pp. 129-146) Mahwah, NJ: Lawrence Erlbaum Associates.

(5)

(6)

Appendix A: Interview Protocol

A. Vice Principal

1. How do you see the NE?

2. What challenge do you face in preparing your students for NE? 3. What programs does school have in preparing students for NE? 4. What is your expectation for your students for the NE ahead?

B. Teachers

1. How do you see the NE?

2. What challenge do you face in preparing your students for NE? 3. How do your students respond to NE?

4. What is your expectation for your students for the NE ahead?

C. Students

1. How do you see NE? 2. What do you feel now?

3. What preparation do you make? 4. What challenge do you face?

5. What supports does your school give?

6. How do your teachers teach you to prepare the NE? 7. What is your expectation for your NE ahead? 8. Have you been ready now?

A study on washback of English national examination among 12th graders of a private religion-based vocational High School in Yogyakarta.

A STUDY ON WASHBACK OF ENGLISH NATIONAL

EXAMINATION AMONG 12

_{GRADERS OF A PRIVATE}

RELIGION-BASED VOCATIONAL HIGH SCHOOL IN

YOGYAKARTA

A STUDY ON WASHBACK OF ENGLISH NATIONAL

EXAMINATION AMONG 12

_{GRADERS OF A PRIVATE}

RELIGION-BASED VOCATIONAL HIGH SCHOOL IN

YOGYAKARTA

Trust in the Lord with all thine heart ;

and lean not unto thine own understanding.

In all thy ways acknowledge him,

and he shall direct thy paths.

Proverbs 3:5-6

I dedicate this thesis to the Almighty Jesus Christ,

my parents, my sister,

Parts

Dokumen yang terkait

TYPE OF READING COMPREHENSION QUESTION IN ENGLISH NATIONAL EXAMINATION OF SENIOR HIGH SCHOOL Type Of Reading Comprehension Question In English National Examination Of Senior High School In 2011-2015.

TYPE OF READING COMPREHENSION QUESTION IN ENGLISH NATIONAL EXAMINATION OF SENIOR HIGH SCHOOL Type Of Reading Comprehension Question In English National Examination Of Senior High School In 2011-2015.

A STUDY ON THE EXPRESSIONS OF POLITENESS IN SENIOR HIGH SCHOOL ENGLISH TEXTBOOKS A Study On The Expressions Of Politeness In Senior High School English Textbooks.

A STUDY ON THE EXPRESSIONS OF POLITENESS IN SENIOR HIGH SCHOOL ENGLISH TEXTBOOKS A Study On The Expressions Of Politeness In Senior High School English Textbooks.

STUDENTS’ DIFFICULTIES IN SPEAKING ENGLISH AND THEIR STRATEGIES TO OVERCOME THE DIFFICULTIES : A Descriptive Qualitative Study of 12th Graders of Software Engineering Programme at a Vocational School in Bandung.

A STUDY ON THE AMBIGUITY FOUND IN ENGLISH EXERCISES OF VOCATIONAL SCHOOL STUDENT’S A Study On The Ambiguity Found In English Exercises Of Vocational School Student’s Exercise Books.

THE EVALUATION SYSTEM OF ENGLISH TEACHING IN THE IMPLEMENTATION OF SCHOOL BASED CURRICULUM (SBC) FOR VOCATIONAL SCHOOL : A Case Study at a Private Vocational School in Soreang, South Bandung, West Java.

A study on the content validity and authenticity of English national examination year 2015 for Junior High School.

Vocational High School English teachers` lived experience on English Computer-based national examination.

A study on washback of English national examination among 12th graders of a private religion based vocational High School in Yogyakarta

Dukungan

Links

A study on washback of English national examination among 12th graders of a private religion-based vocational High School in Yogyakarta.

A STUDY ON WASHBACK OF ENGLISH NATIONAL

EXAMINATION AMONG 12

GRADERS OF A PRIVATE

RELIGION-BASED VOCATIONAL HIGH SCHOOL IN

YOGYAKARTA

A STUDY ON WASHBACK OF ENGLISH NATIONAL

EXAMINATION AMONG 12

GRADERS OF A PRIVATE

RELIGION-BASED VOCATIONAL HIGH SCHOOL IN

YOGYAKARTA

Trust in the Lord with all thine heart ;

and lean not unto thine own understanding.

In all thy ways acknowledge him,

and he shall direct thy paths.

Proverbs 3:5-6

I dedicate this thesis to the Almighty Jesus Christ,

my parents, my sister,

Parts

Dokumen yang terkait

TYPE OF READING COMPREHENSION QUESTION IN ENGLISH NATIONAL EXAMINATION OF SENIOR HIGH SCHOOL Type Of Reading Comprehension Question In English National Examination Of Senior High School In 2011-2015.

TYPE OF READING COMPREHENSION QUESTION IN ENGLISH NATIONAL EXAMINATION OF SENIOR HIGH SCHOOL Type Of Reading Comprehension Question In English National Examination Of Senior High School In 2011-2015.

A STUDY ON THE EXPRESSIONS OF POLITENESS IN SENIOR HIGH SCHOOL ENGLISH TEXTBOOKS A Study On The Expressions Of Politeness In Senior High School English Textbooks.

A STUDY ON THE EXPRESSIONS OF POLITENESS IN SENIOR HIGH SCHOOL ENGLISH TEXTBOOKS A Study On The Expressions Of Politeness In Senior High School English Textbooks.

STUDENTS’ DIFFICULTIES IN SPEAKING ENGLISH AND THEIR STRATEGIES TO OVERCOME THE DIFFICULTIES : A Descriptive Qualitative Study of 12th Graders of Software Engineering Programme at a Vocational School in Bandung.

A STUDY ON THE AMBIGUITY FOUND IN ENGLISH EXERCISES OF VOCATIONAL SCHOOL STUDENT’S A Study On The Ambiguity Found In English Exercises Of Vocational School Student’s Exercise Books.

THE EVALUATION SYSTEM OF ENGLISH TEACHING IN THE IMPLEMENTATION OF SCHOOL BASED CURRICULUM (SBC) FOR VOCATIONAL SCHOOL : A Case Study at a Private Vocational School in Soreang, South Bandung, West Java.

A study on the content validity and authenticity of English national examination year 2015 for Junior High School.

Vocational High School English teachers` lived experience on English Computer-based national examination.

A study on washback of English national examination among 12th graders of a private religion based vocational High School in Yogyakarta

Dokumen yang Anda mencari sudah siap untuk unduhkan

_{GRADERS OF A PRIVATE}

_{GRADERS OF A PRIVATE}