The meaning of the 2015 English national examination to Senior High School students

THE MEANING OF THE 2015 ENGLISH NATIONAL EXAMINATION TO SENIOR HIGH SCHOOL STUDENTS

A THESIS

Presented as Partial Fulfillment of the Requirements for the Degree of Magister Humaniora (M. Hum.)

in English Language Studies

Sabina Thipani

Student Number: 146332021

THE GRADUATE PROGRAM IN ENGLISH LANGUAGE STUDIES SANATA DHARMA UNIVERSITY

i

THE MEANING OF THE 2015 ENGLISH NATIONAL EXAMINATION TO SENIOR HIGH SCHOOL STUDENTS

A THESIS

Presented as Partial Fulfillment of the Requirements for the Degree of Magister Humaniora (M. Hum.)

in English Language Studies

Sabina Thipani

Student Number: 146332021

THE GRADUATE PROGRAM IN ENGLISH LANGUAGE STUDIES SANATA DHARMA UNIVERSITY

A THESIS

THE

MEAMNG

OF

THE

2815

ENGLISH

NATIONAL

EXAMINATION TO

SENIOR

HIGII

SCHOOL STUDENTS

Sabina Thipam

Student Numb er : I 46332021

Approved by

Dr. J. Bismoko

A THESIS

THE MEANING

OF

THE

2OI5

ENGLISH

NATIONAL

EXAMINATION TO

SENIOR

HIGH

SCHOOL

STUDENTS

Presented by Sabina Thipani

Student Number: | 46332021

Defended before the Thesis Committee and Declared Acceptable

THESIS COMMITTEE

Chairperson : Dr. B.B. Dwijatmoko, M.A.

Secretary

: Dr. J. Bismoko

Members

: 1. F.X. Mukarto, Ph.D.

2.Dr. E. Sunarto, M.Hum.

Yogyakarta JuLy 20, 2017 The Graduate Program Director

f+

fhqf

STATEMENT OF ORIGINALITY

This is to certify that all _{the ideas, phrases, and sentences, unless otherwise stated,} are mine.

I understand

the fulI consequences including degree cancellation

if I

took somebody else's ideas, phrases, or sentences without proper references.

Yogfakarta,

LEMBAR PERNYATAAN PERSETUJUAN

PUBLIKASI KARYA

ILMIAH

UNTUK KEPENTINGAN AKADEMIS

Yang bertanda tangan di bawah ini, saya mahasiswa Universitas Sanata Dharma:

Nama

: Sabina Thipani

Nomor

Mahasiswa

: 146332021

Demi pengembangan ilmu pengetahuan, saya memberikan kepada Perpustakaan Universitas Sanata Dharma karya ilmiah saya yang berjudul:

THE MEANING OF THE 2015 ENGLISH NATIONAL EXAMINATION SENIoR HIGH

,T3oo,

STUDENTS

beserta perangkat yang diperlukan (bila ada). Dengan demikian saya memberikan

kepada Perpustakaan Universitas Sanata Dharma

hak

untuk

menyimpan, mengalihkan dalam bentuk media lain, mengelolanya dalam bentuk pangkalan data, mendistribusikan secara terbatas, dan mempublikasikannya di Internet atau

media lain untuk kepentingan akademis tanpa perlu meminta

ijin

dari saya

maupnn memberikan royalti kepada saya selama tetap mencantumkan nama saya

sebagai penulis.

Demikian pernyataan ini yang saya buat dengan sebenarnya. Dibuat di Yogyakarta

Pada tanggal: 13 Juli 2017 Yang menyatakan

ffi

vi ABSTRACT

Thipani, Sabina. 2015. The Meaning of the 2015 English National Examination to Senior High School Students. Yogyakarta: The Graduate Program in English Language Studies, Sanata Dharma University.

The National Examination was always defined by the authorities (the World Bank, the government, critics, teachers, and researchers). Especially by the World Bank and the government, learners are not involved in the decision making and defining process of the National Examination. This situation encouraged me to

conduct research to dig learners‟ lived experience regarding the National

Examination. It asks the meaning of the 2015 English National Examination to senior high school students.

To answer the research question, phenomenology approach is implemented. The texts of this researched were gained through interviews. The setting is Western Indonesia and the participants were Senior High School students which experienced the 2015 English National Examination as students. The nature of this research is hermeneutic phenomenology. In order to find the meaning, the text was analyzed by using the following steps: 1) horizonalization, 2) clustering and thematizing, 3) individual textural descriptions, 4) individual structural description, 5) individual textural-structural description, 6) composite textural description, 7) composite structural description, and 8) synthesis of structural and textural description.

Five themes are found from the text analysis process. They include: 1) others’ concern, 2) independency, 3) graduating as focus, 4) anxiety, 5) self-understanding. In the National Examination process, the participants are affected by parents, teachers, and friends. Most participants in this research considered independency important. Their independency was reflected through their initiative, stable motivation, and autonomy. Not only considering independency important, the participants also considered graduating important. During the preparation for and on the D-day of the examination, anxiety was the feeling which dominated the participants and affected their actions. The National Examination also helped the participants to understand themselves better. Because of the National Examination, they could identify their weaknesses, ways to improve their weaknesses, and aspects they improved because of the National Examination.

The research results showed that the participants‟ way of comprehending the

National Examination is different from the authorities. However, the National Examination still needs to be conducted because the research results show it helped the students sharpen their independency and understand themselves better. Nevertheless, the policy makers, schools, and parents need to make sure the students improve themselves well in the process. Future researchers could involve participants from other levels of education or students who fail the examination. Key words: Meaning, Lived-Experience, the 2015 English Senior High School National Examination.

vii ABSTRAK

Thipani, Sabina. 2015. The Meaning of the 2015 English National Examination to Senior High School Students. Yogyakarta: The Graduate Program in English Language Studies, Sanata Dharma University.

Ujian Nasional (UN) hampir selalu didefinisikan oleh otoritas (Bank Dunia, pemerintah, kritikus, guru, dan peneliti). Terutama oleh Bank Dunia dan pemerintah, siswa tidak dilibatkan dalam proses pengambilan keputusan dan pendefinisian UN. Situasi ini mendorong saya untuk melakukan penelitian yang bertujuan menggali lived experience siswa ketika menjalani UN. Rumusan masalah penelitian ini adalah: Apa makna UN Bahasa Inggris Tahun 2015 bagi siswa SMA?

Untuk menjawab rumusan masalah tersebut, metode fenomenologi digunakan dalam penelitian ini. Teks dalam penelitian ini diperoleh melalui wawancara. Latar penelitian ini adalah Indonesia Barat dan partisipan penelitian ini adalah siswa SMA yang mengalami UN Bahasa Inggris Tahun 2015. Penelitian ini adalah penelian fenomenologi hermeneutik. Untuk menggali makna, teks yang diperoleh dianalisa melalui langkah-langkah berikut: 1) horizonalisasi, 2) pengelompokan dan pentemaan, 3) pendeskripsian struktural masing-masing partisipan, 4) pendeskripsian tekstural masing-masing partisipan, 5) pendeskripsian tekstural-struktural masing-masing partisipan, 6) pendeskripsian struktural seluruh partisipan, 7) pendeskripsian tekstural seluruh partisipan, dan 8) sintesis deskripsi struktural dan tekstural.

Ditemukan lima tema dari proses analisa teks. Kelimanya mencakup: 1) pengaruh orang lain, 2) independensi, 3) lulus sebagai tujuan, 4) kecemasan, 5) pemahaman diri. Partisipan juga dipengaruhi oleh orang tua, guru, dan teman mereka. Sebagian besar partisipan dalam penelitian ini menganggap independensi penting. Independensi mereka terlihat dari inisiatif mereka, independensi mereka, dan kemandirian mereka. Selain independensi, partisipan juga menganggap kelulusan sebagai hal penting. Selama persiapan dan pengerjaan UN, kecemasan adalah perasaan yang mendominasi partisipan dan mempengaruhi tindakan-tindakan mereka. UN juga membantu partisipan memahami diri mereka dengan lebih baik. UN membantu mereka mengetahui kekurangan mereka membantu mereka meningkatkan kelemahan mereka, membantu mereka menemukan strategi untuk menghadapi UN, dan membantu mereka mengidentifikasi aspek-aspek dalam dirinya yang mengalami peningkatan karena UN.

Hasil penelitian menunjukkan bahwa partisipan memaknai UN secara berbeda dengan otoritas. Meskipun demikian, UN tetap perlu dilakukan karena hasil penelitian ini menunjukkan bahwa UN membantu siswa mengasah independensi mereka dan memahami diri mereka dengan lebih baik. Namun, pembuat kebijakan, sekolah, dan orang tua perlu memastikan bahwa siswa dapat mengembangkan diri mereka dengan baik dalam proses UN. Peneliti di masa mendatang dapat mengeksplorasi partisipan dari level pendidikan yang berbeda atau partisipan yang gagal dalam UN.

viii

ACKNOWLEDGMENTS

I would like to first of all thank my thesis advisor, Dr. J. Bismoko, for his guidance, patience, inspiration, and critical insights, not only during my research but also during my study in KBI. The energy he has always put when teaching and guiding us has always motivated me to be a better educator and researcher. I am also thankful for the chance he gave us to get acquainted with phenomenology, a study which helped me answer a lot of questions I had as a student.

I would also like to express my deepest gratitude to Dr. B. B. Dwijatmoko, M.A., F.X. Mukarto, Ph. D, and Dr. E. Sunarto, M. Hum for the constructive criticism and insights during the process of thesis review and thesis defence. I am also thankful that I have the chance to be taught by all lecturers in KBI. Their guidance had helped me to understand English language education study more comprehensively and to develop discipline in learning and researching.

My warmest thanks also go to my family: my mother, my father, and my brother. I thank them for the support, love, and inspiration they always give in every step of my life. I would also like to thank Wahmuji for our companionship, our conversations and insightful criticism he gave during my research and for his patience and support during the hard times.

I would also like to express my sincere gratitude to friends in KBI for the time we spent and the process we had during our study. I hope what we have learnt from each other will help us to grow in the next phase of our life.

Last but not least, I would like to express my deepest gratitude to all my participants for their willingness to share experiences and their cooperation during the research. Their experiences have helped me learn more and understand not only other humans but also myself. I owe them a lot and wish them all the best.

ix

TABLE OF CONTENT

TITLE PAGE... i

APPROVAL PAGE... ii

THESIS DEFENSE APPROVAL... iii

STATEMENT OF ORIGINALITY... iv

STATEMENT OF LICENSE AGREEMENT... v

ABSTRACT... vi

ABSTRAK... vii

ACKNOWLEDGEMENTS... viii

TABLE OF CONTENTS... ix

LIST OF TABLES... xii

LIST OF APPENDICES... xiii

CHAPTER 1 INTRODUCTION... 1

A. BACKGROUND... 1

B. PROBLEM IDENTIFICATION... 10

C. PROBLEM LIMITATION... 11

D. PROBLEM FORMULATION... 11

E. RESEARCH GOALS... 11

F. RESEARCH BENEFITS... 12

CHAPTER II LITERATURE REVIEW... 14

A. THEORETICAL REVIEW... 14

1. Assessment and Test... 14

a. Assessment... 14

b. Test... 16

2. National Assessment... 17

3. National Examination... 21

a. Research on the National Examination... 21

b. Criticism of the National Examination... 25

1) The National Examination to Increase Intelligence and Improve Education Quality... 25

2) The National Examination Widen the Academic Achievement Gap... 26

3) An Incomprehensive Test... 28

4) The National Examination Had the Potential to Increase Dropping Out Risk... 29

5) Pressure on Students and Teachers... 30

6) Cheating... 31

7) On Mapping Education Quality... 33

c. Recommendation for a Better Assessment System... 34

1) The Importance of Evaluating the National Examination... 34

2) The Importance of Evaluating the Government... 34

3) A Better Assessment System... 36

4) How to Determine Students‟ Graduation... 36

d. The 2015 National Examination in Indonesia... 37

4. Senior High School... 39

x

a. Development of Phenomenology... 39

b. The Nature of Phenomenology... 40

c. Narrative and Hermeneutic Phenomenology... 42

d. Important Concepts in Phenomenology... 43

1) Consciousness... 43

2) Act... 43

3) Perception... 44

4) Intentional Experience... 44

5) Intersubjective Validity... 45

6) Reduction... 45

7) Lived Experience... 46

8) Meaning... 47

9) Empathy... 49

10)Reflection... 49

11)Theme... 51

e. Research on Lived Experience... 51

f. Phenomenological Research and Research on Perception... 56

B. THEORETICAL FRAMEWORK (PRE-ASSUMPTION) 59 CHAPTER III METHODOLOGY... 66

A. METHOD... 66

B. RESEARCH DESIGN... 66

1. Research Setting... 67

2. Research Participants... 67

C. TEXT... 68

D. DESCRIPTION AND INTERPRETATION... 69

1. Horizonalization... 69

2. Invariant Constituent Clustering and Thematizing... 69

3. Individual Textural Descriptions (Stories) ... 70

4. Individual Structural Description (Interpretations)... 70

5. Individual Textural-Structural Descriptions (Stories and Interpretations) ... 71

6. Composite Textural Description (Stories) and Composite Structural Description (Interpretations)... 71 7. Synthesis of Textural and Structural Description (Meaning)... 72

E. TRUSTWORTHINESS... 72

CHAPTER IV DESCRIPTION AND INTERPRETATION... 75

A. INDIVIDUAL TEXTURAL DESCRIPTIONS (STORIES) 75 1. Dylan‟s Textural Description (Story)... 75

2. Baez‟s Textural Description (Story) ... 79

3. Sting‟s Textural Description (Story) ... 84

4. Dolores‟ Textural Description (Story) ... 88

B. INDIVIDUAL STRUCTURAL DESCRIPTIONS (INTERPRETATIONS) ... 94

1. Dylan‟s Structural Description (Interpretation)... 94

2. Baez Structural Description (Interpretation)... 99

xi

4. Dolores‟ Structural Description (Interpretation)... 107

C. INDIVIDUAL TEXTURAL-STRUCTURAL DESCRIPTIONS (STORIES AND INTERPRETATIONS)... 109

1. Dylan‟s Textural-Structural Description (Story and Interpretation) ... 109

2. Baez‟s Textural-Structural Description (Story and Interpretation) ... 112

3. Sting‟s Textural-Structural Description (Story and Interpretation) ... 114

4. Dolores‟ Textural-Structural Description (Story and Interpretation) ... 117

D. COMPOSITE TEXTURAL DESCRIPTION... 120

E. COMPOSITE STRUCTURAL DESCRIPTION... 124

F. TEXTURAL-STRUCTURAL SYNTHESIS (MEANING)... 128

1. Themes... 129

a. Others‟ Concern... 130

b. Independency... 131

c. Graduating as Focus... 132

d. Anxiety... 132

e. Self-understanding... 133

2. Summary... 133

CHAPTER V CONCLUSIONS... 136

A. RESEARCH RESULTS... 136

B. IMPLICATIONS... 138

C. RECOMMENDATIONS... 139

BIBLIOGRAPHY... 141

xii

LIST OF TABLES

Table 2.1.: The Focus of the Recent Research Compared with the Previous

Research Reports... 55 Table 2.2.: The Government and Critics/Teachers on the National

xiii

LIST OF APPENDICES Appendix 1

Interview Questions... 147

A. General Main Questions... 147

B. Follow Up and Probing Questions for All Participants... 148

C. Follow Up and Probing Questions for Dylan... 148

D. Follow Up and Probing Questions for Baez... 152

E. Follow Up and Probing Questions for Sting... 153

F. Follow Up and Probing Questions for Dolores... 155

Appendix 2 Invariant Constituent Clustering and Thematizing... 160

A. Dylan‟s Invariant Constituent Clustering and Thematizing... 160

B. Baez‟s Invariant Constituent Clustering and Thematizing... 164

C. Sting‟s Invariant Constituent Clustering and Thematizing... 172

D. Dolores‟ Invariant Constituent Clustering and Thematizing... 177

Appendix 3 Additional Elaboration on the 2015 National Examination in Indonesia... 184

1

CHAPTER I INTRODUCTION

This chapter is to describe the justification and feasibility of the research project. From discussion on the relevance and benefit in all sub chapters, the justification of this research is reflected. The feasibility of this research can especially be seen under the discussion of Background, Problem Identification, Problem Limitation, and Problem Formulation.

A. BACKGROUND

In the popular context, the National Examination was always defined by the

authorities, especially the World Bank and the government. Learners or students‟

opinions were almost never considered. For the World Bank, the National Examination, or what was called national assessment in the more universal context, was merely one of the tools they needed to have in order to achieve inclusive and sustainable globalization. For the government of Indonesia, it was a great assessment which they thought would help students to improve motivation and achievement, and also the schools and education system to improve their quality. Contradictory to the World Bank and the government, Indonesian critics and teachers saw the National Examination as an assessment which was not comprehensive, which could increase dropping out risk, which gave pressure to teachers and students, and thus needed to be evaluated and be reformed into a better assessment system. The critics were also more sympathetic to students than the World Bank or the government. Although in the 2015 National Examination

2

the government seemed to listen to teachers and critics more, learners‟ thought

were still not seen as one of the sources used in policy or decision making.

When I was a student, I found this situation hard. I remember feeling nervous and scared in Junior High School since I knew that the National Examination

would be used to determine my and other students‟ graduation. I thought the

burden felt heavier especially because I was always among the best three in class. Because I was always among the best three I thought it would be so embarrassing if I did not pass the National Examination.

Not so long after the National Examination was over, the headmaster of the school called me. She told me that I passed the examination, but not with my own real scores. None of the students in my school passed if the school used their real scores. My problem was not in my average score. It was one of my scores which was less than what was standardized. She told me that she lobbied the authority in

Education Office to fake my scores and other students‟ so that we could pass.

When I knew the fact, I felt shocked and sad. I was not glad even though I passed the examination. I felt incapable, not deserving the graduation. I felt traumatic ever since, felt that I could never be able to be good again academically. I did not thank nor protest the decision taken by the school to all of us because I thought I was powerless, I was just a student.

When I was in Senior High School, I have never got rid of the National Examination. Its function to determine graduation always haunted me during my study. Even so, the burden I had in Senior High School was not as heavy as the burden I felt in Junior High School. Since my Senior High School was one of the best schools in town, the atmosphere was competitive. It was hard to maintain to

3

be even the best ten in class. However, I knew that not passing the National Examination was a huge embarrassment for us. I still felt scared, panic, and most of all mad with the fact that the National Examination was used to determine my graduation.

The combination of these feelings, especially the anger, made me keep complaining during my study. I often shared my anxieties to friends and my mother. I complained about how unfair the examination was for me and other students. The examination was unfair because it determined my three-year process of learning in only three days. The examination only asked 50 questions for each subject. Meanwhile, I learnt so much more. I did not think the examination could be used to tell the world if I was a good student or not. I also complained about how the government misunderstood me as a student. On television and newspaper, the government kept saying about how great the examination was and how important it was for the students. Meanwhile, at the same time I did not think they knew the anxieties I had as a student. I felt my life was defined and controlled by the people who could not sympathize with me or other students at all. I complained almost all the time but I did not know how the complaints could be heard. I thought I was just a student. There was nothing I could do.

The combination of the feelings I mentioned above, especially the panic, made me decide to join a study course. I have never joined any courses before because I could study better if I could understand things by myself. I could study better if I could manage everything by myself. After I finally joined the course, I indeed did not feel comfortable studying. In the course I was always told what to do. We mostly studied samples of examination questions. The teachers there taught me

4

how to pragmatically solve the questions. It bored me because I felt studying was fun if I could fix things myself and know the basic and concept map of the subject. I had to adapt more because studying in the course was very different from studying by myself. Eventually, after joining the course my scores decreased drastically.

Another impact of the feelings I had during my study in Senior High School was me ended up cheating. Other than all of those feelings, at that time I had some other justification. Some of them were related to the arguments I had mentioned above. I thought the government was unfair. Thus, it was also fine for me to do the examination unfairly. I also thought, because I did not have any access to express my opinion and anger, cheating was the only room left for me to express it. What I meant by cheating, though, was not depending the whole examination to others. When preparing for the examination I heard there would be leaks of examination answers. I eventually knew that some of my friends bought them and speeded them to the whole school. Almost all students used them. I used the leaked answers only as reference. There were a couple of versions of answers. I did all examination questions by myself and matched them with the leaked answers. When they were different I rechecked my work again.

As a student I thought graduating was important. However, I thought it would be fair if graduating was determined by my three-year studying process in Junior or Senior High School. Since the graduation was determined by the National Examination, an examination which I thought not considering process, I felt burdened with it. During my study I, most of the time, felt scared and panic; haunted. I even decided to join a course because of the mixed feeling I had even

5

though I prefer studying alone. I even decided to cheat even though I have never cheated before.

As a student I also felt powerless. When my Junior High School headmaster told me that my scores were faked, I did not react. I also had intense anxiety especially when I was in Senior High School. I felt the examination was unfair and I felt misunderstood by the authority. However, I did not do anything than just sharing the opinion to friends and my mother. I did not do anything than just cheating to express my anger. I did not do anything because as a student I felt powerless.

My lived experience encouraged me to conduct research on others‟ lived experience on the National Examination. By conducting research on the topic, I hope to give the chance to learners or ex-learners to deliver their lived experience to important parties in education world. As a learner, I felt I never had the empathy I needed when dealing with the National Examination. Thus, I also hope

the research will evoke teachers‟, critics‟, government‟s and policy makers‟ empathy on learners‟ experience regarding the National Examination. Other than

that, by conducting this research I hope I will be able to identify myself with the

participants‟ experience and to be able to have more scientific understanding on my and others‟ lived experience regarding the National Examination. I also hope I

will be able to enhance my ability in digging and delivering my and others‟ lived experience on the National Examination scientifically.

The lived experience on the National Examination that I had was experienced when I was in junior and senior high school. However, in this research I chose to focus more on senior high school. Senior high school level was chosen because,

6

first, I considered senior high school students have already had richer experience on the National Examination since they had experienced the examination for three times (in elementary school, in junior high school, and in senior high school). The second reason is because I consider senior high school students have better capability to articulate their lived experience compared to students in lower levels. It is because hey are more mature compared to students from lower levels.

As stated previously, the authorities had not involved learners in their arguments regarding the National Examination. Different attention to learners was given in academic world. The attempt which academicians made to have learners‟ involvement in their research can be seen in three of six research reports regarding

the National Examination I found. The first research discovered one student‟s

lived experience on the English National Examination (Fiharsono, 2011). It

discovered four themes regarding the topic: (1) student‟s understandings, (2) student‟s beliefs, (3) student‟s intentions, and (4) student‟s action. The second

research dug six qualities of useful language tests in the 2012 English National Examination (Thipani, 2013). The findings involved the 2012 English National

Examination‟s reliability, validity, authenticity, interactiveness, impact, and

practicality. The third research questioned the washback experienced by the twelfth graders of a private religion-based vocational high school in Yogyakarta regarding the National Examination preparation (Subagyo, 2014). The research showed that the National Examination encouraged the students to be more

prepared with “potential challenges such as pressure and time constraints” and those teachers could “comprehend the phenomenon faced by their students”

7

Other than the three research reports, I found three other research reports discussing the National Examination. The first research discussed the relevance between the Senior High School National Examination multiple choice reading items with objectives of English reading skills in KTSP (Merina, 2009). The research showed that the National Examination only covered two cognitive processes of Anderson taxonomy (remember and understand) and did not cover all cognitive processes as required in KTSP. However, the National Examination had covered two types of written text in KTSP (short functional text and simple essays). The research also showed that vocabulary mastery was measured even though it was not required in KTSP. The second research discovered EFL

teachers‟ voices regarding the National Examination (Siswanti, 2010). The

researched showed the participants‟ different opinion how the National

Examination should be conducted. One of the participants thought the National Examination should be conducted the way it had been conducted meanwhile another one thought the National Examination should be conducted by adjusting to vocational high school students‟ competencies. However, both of them thought the National Examination should be based on English for Specific Purposes. The last research discovered content validity and authenticity of the 2012 Senior High School English National Examination (Widyaningrum, 2014). The research showed that the 2012 Senior High School English National Examination was 98.8% valid in content and 79.5% authentic.

I found ten research reports discussing English acquisition with phenomenology. Only one research discusses English education design. Another one focused on evaluation. From eight research reports on implementation, six of

8

them try to discover meaning from the perspective of the learners. The rest two focused on meaning from the perspective of the teachers.

The first research on design discovered the meaning of textbook selection to SMP English teachers (Kiswara, 2009). The research showed that all participants

“tried to meet the students‟ learning needs, interest and cultural background”

(Kiswara, xiv) in the process of textbook selection, even though the focus they have regarding learning needs was different. The participants also had different level of autonomy which affected their perceptions and the end their process of selecting textbooks.

The second research was elaborated on the discussion on the National Examination above. It was written by Fiharsono (2011).

The third research, which focused on implementation, discussed the meaning of the children lived experience in Turkey on learning English through computer games (Turgut & Irgin, n.d.). The finding shows that the students found benefits (including transfer, motivation, and awareness) by learning English through computer games (Turgut & Irgin).

The fourth research, also focusing on implementation, discussed Colombian adult English language learner selection and using of language learning strategies (Paredes, 2010). The research showed that the learners used learning strategies

which are not included in Oxfords‟ taxonomy (Paredes).

The fifth research discovered the meaning of learning English to Theology

students (Trisnowati, 2011). The research finding showed five themes: “respect

for language use, reflection on language learning experience, self-knowledge, professional development, and self-actualization” (Trisnowati, xv). From the

9

themes she found two essences. The first one is “meaningful language learning in the sense of respect for language use” and the second one is “a desire in seeking

for understanding of the meanings of life through language learning in theology

studies” (Trisnowati, xv).

The sixth research discussed the meaning of blog-assisted learning in the ESL writing classroom to Taiwanese students (Lin, Groom, & Lin, 2013). The research showed that blog-assisted ESL writing learning made students with low proficiency levels feel anxious and embarrassed of the reactions from other students regarding their works (Lin, Groom, & Lin).

The seventh research discovered the meaning of vocabulary learning using IPALL to students (Sastrikirana, 2015). From the research she found empirical

themes (“better knowledge of word meaning and forms, better learning

atmosphere, chances to repeat, focus and concentration” (Sastrikirana, xvii)) and

transcendent themes (“perseverance, honesty, acceptance, connection with a

bigger world, God talks to me” (Sastrikirana, xvii)).

The eighth research discovered the meaning of leaning English to International Class students of University of Muhammadiyah Yogyakarta (UMY) (Stania, 2015). She found two interpreted themes which are related to body and mind and to God.

The ninth research discovered the meaning of lived experience of “becoming

and being a woman teacher” and women experience on teaching boys and girls

(Kirk, 2004). The research result is description of the meaning of experience of becoming and being a woman teacher and women experience on teaching boys and girls (Kirk).

10

The last research discovered the meaning of teaching English large classes to a

novice teacher (Astuti, 2010). Specifically she discovered what “the teacher

thinks about teaching English large classes, how she gives meaning to it, how she conducts and approaches her teaching, how she experiences success and failure, what she learns from it to become more empowered and self-fulfilling” (Astuti, xiv).

The research I conducted aim to discover the meaning of the lived experience on the 2015 English Senior High School National Examination to the Senior High School students. Thus it is expected to contribute to lived-experience studies on English Examination in particular and English evaluation in general from the perspective of the learners. In the broader scope, this research is expected to enrich discussion in English acquisition study.

B. PROBLEM IDENTIFICATION

This research aims to discover the meaning of the 2015 English National Examination to Senior High School students. The English National Examination is a smaller part of English assessment. Meanwhile, English assessment is a type of English evaluation. In English education, English evaluation is one of the divisions other than concept, design, implementation, and revision. English education itself is part of English acquisition. This research is thus, as stated previously, a research on a smaller part of English acquisition.

Other than that, since the research aim to discover meaning, this research can be identified as a phenomenology research. More explanation on this can be seen in Chapter II and III of the research.

11

C. PROBLEM LIMITATION

One of the aspects of the research context is the National Examination. This research will thus be mostly relevant to audiences in Indonesia even though there is also possibility that the research may have relevance to broader audiences who experience or are involved in or exposed to national assessment.

Another aspect is 2015. This research will thus be actual mostly to those involved in the National Examination near the year 2015 in the future. However, there is possibility that the research can be relevant to the need of the next national examinations in the further future.

Senior High School is also one of other aspects of the problem. This research will thus be mostly relevant to those involved in the Senior High School National Examination. However, there is possibility that the research can be useful to other levels of the National Examination.

Not only that, the research is a phenomenological research. This research thus has strength in its depth. However, it is weak in its width.

D. PROBLEM FORMULATION

This research aims to discover the meaning of the lived experience on the 2015 English Senior High School National Examination to the Senior High School students. In order to reach the aim, I proposed the following research question: What is the meaning of the 2015 English National Examination to senior high school students?

E. RESEARCH GOALS

The immediate goal of the research is to discover the meaning of the lived experience on the 2015 English Senior High School National Examination to the

12

Senior High School students. The intermediate goal is to contribute to the discussion on the National Examination in particular and evaluation in general. This research also aims to enrich discussion on phenomenological research.

The ultimate goals of the research are more reflective, illuminating, empathic, self-actualizing, and self-fulfilling individuals. From the research, the participants and the researcher are demanded to reflect on their and others‟ lived experience. By reflecting, the participants and researcher are illuminated. From the research, the participants and the researcher can also actualize themselves. The participants actualize themselves by expressing their textural reflection while the researcher actualizes herself by, as stated previously, digging phenomenological sensitivity,

exercising hermeneutic ability, and exploring the participants‟ textual reflection.

From the research, the audience can understand others‟ lived experience and are expected to have empathic understanding on it. Empathic understanding was the key to better equity. With better equity, individuals can be more autonomous. Autonomy brings us to empowerment and empowerment makes use be more self-actualized, and in the end, more self-fulfilling.

F. RESEARCH BENEFITS

The research can bring benefits to the research participants, researcher, and

audience. The research participants‟ benefits from having the chance and a

channel to deliver textual reflection on the lived experience they have. By doing this, they have the chance to be illuminated and to actualize themselves.

Meanwhile, the researcher can actualize self by experiencing pedagogical process of digging phenomenological sensitivity, exercising her hermeneutic

13

ability, and having the chance to explore others‟ textual reflection. This process

also brings the researcher to illumination.

The audience, on the other side, which includes students, teachers, critics, or policy makers and the government, benefits from enhancing their empathic understanding to learners and from having materials they can use as considerations to their actions. Students can identify themselves with the research results and reflect their actions with them, and use the reflection to act with more consideration. Teachers can use the research results as reflection materials and to use the reflection to teach more empathetically. Critics can use the research results as reflection and insights to their criticisms. Similarly, policy makers and the government can use the research results as reflection and insights to their policies.

14

CHAPTER II LITERATURE REVIEW

This chapter is to discover the logical truth of the research. In the Theoretical Review, relevant concepts of the study, which includes phenomenology and the national examination pre-understanding, are presented. In the Theoretical Framework, pre-assumption from the concepts are presented.

A.THEORETICAL REVIEW

In this part, universal truth of the relevant concepts and concepts relation are exposed. The concepts meant are related to phenomenology and the National Examination. This part is divided into five more parts: Assessment and Test, National Assessment, National Examination, Senior High School, and Phenomenology.

1. Assessment and Test

The following description contains discussion on assessment and test. It also contains discussion on the relation between the two terms with National Assessment and National Examination.

Further discussion on the National Assessment can be seen after the discussion on assessment and test. Meanwhile, further discussion on National Examination can be seen after the discussion on National Assessment.

a. Assessment

Assessments is a broad term defined as a process for obtaining information that is used for making decisions about students; curricula, programs, and schools; and educational policy. When we say we are “assessing a student‟s competence”, for

15

example, we mean we are collecting information to help us decide the degree to which the students has achieved the learning targets (Nitko and Brookhart, 2011, 3). Scheerens, Glas, & Thomas (2003) define assessment as the information gathering on the achievement of individual student. In the context of language teaching, assessment can be defined as a process of collecting information about

aspects of test taker‟s language ability systematically, on the basis of substantive

ground (Bachman & Palmer, 2012).

Assessment can thus be defined as an information gathering process about individual student language achievement/ability/competence which is done systematically, based on certain substantial basis. The term „assessment‟ in the

discussion under subtitle „National Assessment‟ refers to this definition. The

system which is supported by National Assessment is the system created by the World Bank in its member countries in order to reach its vision.

From the discussion under subtitle „National Assessment‟ it can be concluded that National Assessment belongs to analytical assessment. Analytical assessment provides
specific
feedback along
several
dimensions (Lane, 2008). It breaks

down the performance into different levels of behavior expected, assigning each a point value (which can be weighed if desired), and which are tolerated for a quantitative measure (Blaz, 2001). In other words, it is an assessment which measures several dimensions, each of which with single score, which are broken down from a general performance quantitatively.

National Assessment also belongs to general assessment. General assessment uses criteria and descriptions of performance that generalize across (can be used with different tasks) (Brookhart, 2008).

16

National Assessment also belongs to educational assessment. It is because its characteristic is similar to educational assessment, which is “integral to the

educational process” (construct) (Chatterji, 2003, 2).

National Assessment is also a summative assessment because it has the

characteristics of summative assessment. Summative assessment “aims to

measure, or summarize, what a student has grasped, and typically occurs at the end of a course or unit of instruction” (Heritage, M., 2010, 6).

National Assessment is also related closer to formal assessment. Formal

assessment is “a formal, systematic, summative information-gathering procedures

that are used to make decision about pupil grades, promotion, placement, or honors. (Airasian, 1991, 191).

b. Test

A test is “a method of measuring a person‟s ability, knowledge, or performance

in a given domain” (Brown, 2004, 3). Method is “instrument (a set of techniques,

procedures, or items)” (Brown, 3). It “must be explicit and structured” (Brown, 3).

Measure refers to result offering to test takers (Brown). Individual refers to who the test taker is, her/his experience and background, and her/his abilities (Brown).

Performance implies test taker‟s ability/competence (Brown, 3) while given

domain refers to measurement of “the desired criterion and not include other factors inadvertently” (Brown, 4).

A test can also be defined as a type of assessment in which formal tasks are presented to pupils to obtain systematic evidence about their performance (Airasian, 1991). The primary goal of achievement testing is to provide a fair and

17

representative indication of pupil learning based on the instruction provided. (Airasian).

Test can thus be defined as a method to measure student‟s competence in certain domain formally. The term „examination‟ in „National Examination‟ refers to this definition.

National Examination can be categorized as paper-and-pencil language test. Paper-and-pencil language test “take the form of the familiar examination

question paper” (McNamara, 2000, 5). It is “typically used for the assessment either of separate components of language knowledge (grammar, vocabulary, etc.)

or of receptive understanding (listening and reading comprehension” (McNamara,

5). The test items in this type of test has “fixed response format” (McNamara, 5).

Multiple choice is one of the most important format (McNamara).

National Examination can also be categorized as achievement test.

Achievement test is a test which is “associated with the process of instruction”

(McNamara, 2000, 6). It is usually conducted at the “end of course tests, portfolio assessments, or observational procedures for recording progress on the basis of classroom work and participation” (McNamara, 6).

2. National Assessment

One of the themes which World Bank sets to promote an inclusive and sustainable globalization is World Bank role as a unique and special institution of

knowledge and learning. This theme requires World Bank to “focus continually and rigorously on results and on the assessment of effectiveness” (Greaney & Kellaghan, 2008, ix). This assessment includes assessment in education world. How is World Bank and education related? Education is considered important by

18

World Bank in economic development since economic development requires human knowledge and skills (Greaney & Kellaghan). National assessment provides the information regarding individual student achievements in knowledge and skills which could be useful for World Bank (Greaney & Kellaghan).

Not only for World Bank, information on knowledge and skills is also important for governments and students. Higher levels of knowledge and skills are needed if students would like to participate in the working world in the future (Greaney & Kellaghan, 2008). Skillful and knowledgeable workers are what the government needs in order to compete successfully in globalization (Greaney &

Kellaghan). Other than that, governments also “need to monitor achievement

levels to determine how changes in enrollment and budgetary conditions affect the quality of learning” (Greaney & Kellaghan, 18). National assessment has helped ministries of education to “describe national level of learning achievement, especially in key subject areas, and to compare achievement levels of key sub groups” (Greaney & Kellaghan, 2). It has also been useful to help the ministries

“to support or refute claims that standards of student achievement are rising or falling over time” (Greaney & Kellaghan, 2).

National assessment is also claimed to be significant for policy makers, politicians, and educational community. It is considered effective in providing feedback regarding significant measures to these parties (Greaney & Kellaghan, 2008). It provides information on aspects of the system and basic inputs needed by the system for policy making, information which the government often lacks (Greaney & Kellaghan). Its results can also be used “to change practice in the

19

classroom” (Horn, Wolff, and Velez 1992 as paraphrased in Greaney &

Kellaghan, 19).

However, World Bank considers assessment of effectiveness in education field as the biggest challenge (Greaney & Kellaghan, 2008). This includes national assessment. Even though is considered plays important roles in giving information regarding individual student achievements in national level, not many countries

systematically monitors the learning outcomes “either through conducting their

own assessments of student achievement or through participating in regional or international assessments” (Greaney & Kellaghan, ix-x). The information gathered from the national assessments which have been conducted in some developed countries has not been maximally used (Greaney & Kellaghan). The problems mentioned above might be caused by: the policy makers who are not dedicatedly and committedly involved in the assessment and unwell-communicated assessment analysis results (Greaney & Kellaghan). Another reason is national assessment which is only used symbolically as a scientific and modern tool to legitimate state action thus it was not seriously used in

“management of the education system or in policy making“ (Greaney &

Kellaghan, 21).

Indonesian government has been implementing national assessment since 1950. The first term used to call national assessment was Ujian Penghabisan (1950-1960) (Cessnasari, 2005). It was then called Ujian Negara (1965-1971) (Cessnasari). The term was then changed again into EBTANAS (Evaluasi Belajar Tahap Akhir Nasional) (1980-2001) (Cessnasari, 2005). In 2002 it was called UAN (Ujian Akhir Nasional) (Cessnasari, 2005). In 2003, the national assessment

20

was centralized (Rachman, 2015, April 13). There was a national body preparing questions (Rachman, 2015). This year the government set a minimum passing grade, which would be set higher periodically (Rachman, 2015). The grades were used to determine whether the students could graduate from certain education level or not. Since 2005 the term has been known as UN (Ujian Nasional/National Examination). The term UN has been used until now (2017). In 2011, not only the National Examination was held. Local School Examination (Ujian Sekolah/US) was also held at the end of each level. To decide whether a student could graduate or not, the government combined the National Examination score (worth 60% to the total score) with School Examination score (worth 40% to the total score).

“According to Peraturan Kementerian Pendidikan Nasional Nomor 34 Tahun

2007 (The Law of the Ministry of Education Number 34 Year 2007)”, the

National Examination functions as 1) the tool to map the quality of students and schools and/or educational program, 2) select university student candidates, 3) determine whether students pass or fail from school and/or educational program, and 4) support and help schools to improve the quality of education (Sarasvati, 2012, 230). The National Examination was “held by Badan Standar Nasional Pendidikan (BSNP)” (Sarasvati, 2012, 228). It was “the only body that is allowed to hold the national examination in Indonesia” (Sarasvati, 2012, 228). The

examination was compulsory (Sarasvati). It was “held at the end of each level of

study“ (for students in the 6th grade of elementary school, 9th grade of secondary

school, and 12th grade of high school) (Sarasvati, 229). The format of the

examination was multiple choices (Sarasvati). The students used “a special paper

21

computational device” (Sarasvati, 229). As stated previously, since 2003, the students must pass certain grade in order to graduate from certain level of study (Sarasvati). If students failed the examination, they could “join a remedial

examination” (Sarasvati, 230). It was “held a week after the notice” (Sarasvati,

230). If students still failed, up to 2012, Sarasvati stated that there was no clear procedure that the students could follow in order to pass the examination. Sometimes the students were suggested to keep continuing the study in the higher level as long as in the first year they could get certificate showing they had passed the examination (Sarasvati). Sometimes they are suggested to join PKBM and finish their study there (Sarasvati).

This part elaborates concepts related to National Examination. It is divided into three parts: national assessment, national assessment in Indonesia (before 2015), and the 2015 national examination in Indonesia.

3. National Examination

a. Research on the National Examination

Previous research on the National Examination and the English National Examination discussed washback/impact, validity, authenticity, practicality,

reliability, interactiveness, administrability, student‟s action and teachers‟ voices.

All topics were discussed by six researchers. Three of them used (Merina, 2009; Thipani, 2013; Widyaningrum, 2014) content analysis method. One of the three (Thipani, 2013) used both content analysis and sample survey. One (Siswanti, 2010) used progressive qualitative approach. One research (Fiharsono, 2011) used phenomenology and another one used (Subagyo, 2014) case study. I will describe all those research reports not one by one but per topic.

22

Research on the National Examination washback was conducted by Subagyo

(2014). In his research he had two questions: (1) “What washback do the twelfth

graders of a private religion-based vocational high school in Yogyakarta experience in response to the National Examination preparation?” and (2) “How does the washback happen to the students, teachers, and the school?” (Subagyo, vii). In his research he found that secondary students were more aware of the impacts of the National Examination. He also found that the National Examination encouraged the students to be more prepared with “potential challenges such as pressure and time constraints” (Subagyo, vii). His research also

showed that teachers could “comprehend the phenomenon faced by their students”

(Subagyo, vii). In my research on 2013 I also discussed it even though I used the

term „impact‟. My research showed that “the test developer goals were in accord with the society/education goals” but “did not bring significant impact to the

students and teachers” (Thipani, vii). Fiharsono‟s research also showed the

National Examination washback to his participant. The washback was seen from

the participant‟s action to study previous tests and predictions (2011).

Research reports on the National Examination validity were conducted by Merina (2009), Fiharsono (2011), Thipani (2013), and Widyaningrum (2014). In her research Widyaningrum showed that 98.8% of the 2012 English National Examination content was valid. My research on 2013 showed that the 2012 English National Examination was unconstructively valid. Not only discussed validity in general like the two previous research reports, Merina (2009) specifically discussed content validity in her research. In her research she asked

23

Examination test for senior high school students measure the objectives of English

reading skills as stated in KTSP?”, (2) “Did the texts used in the National

Examination test for senior high school students represent the written text types

mentioned in KTSP?”, and (3) “Did the question types used in the multiple-choice

reading items of the National Examination test for senior high school students

measure the students‟ comprehension through various question types?” (Merina,

5). The research showed that the National Examination only covered two cognitive processes of Anderson taxonomy (remember and understand) and did not cover all cognitive processes as required in KTSP. However, the National Examination had covered two types of written text in KTSP (short functional text and simple essays). The research also showed that vocabulary mastery was measured even though it was not required in KTSP. Quite different from Merina,

through the description of student‟s understandings and student‟s beliefs

Fiharsono (2011) explained not only the English National Examination‟s content

validity, but also the English National Examination‟s criterion-related validity in

his research. According to Fiharsono, the English National Examination did not represent all English competencies listed in the curriculum. Its content validity was low. He also showed that the result of the English National Examination was significantly different from other measurements conducted either at the same time or in the future. Thus, the criterion-related validity was low.

Three research reports (Fiharsono, 2011; Thipani, 2013; Widyaningrum, 2014)

discussed authenticity. Fiharsono‟s research showed that the competencies tested

in the English National Examination are not relevant to English competencies needed in real life. Thus, the examination authenticity was low. My research also

24

showed that the 2012 English National Examination was “not utterly authentic”

(Thipani, vii). Meanwhile, Widyaningrum‟s research showed that the 2012 English National Examination was 79.5% authentic.

My research in 2013 discussed the 2012 English National Examination practicality, reliability, and interactiveness. The research showed that the

examination was “practical in the context of human resources and time allocation

availability but impractical in the context of material resources availability” (Thipani, vii). It also showed that the exam was “reliable in the context of

inter-rater and not reliable in the context of student, test administration, and test”. The

research also indicated that the examination could be considered interactive

because the students‟ level and type of general education, their types and amount

of preparation, and their language ability helped them to be involved in the test. However, it could also be considered not interactive because the students‟ family background, topical knowledge, affective schemata, and strategic competence did not help them to be involved in the test.

Fiharsono‟s research discussed the National Examination administrability and

student‟s action (2011). There were dishonest practices during test administration. One of them was student buying test leak. Discussion on student‟s action include

Fiharsono‟s participant which did all intentions he had during the National

Examination.

Siswanti discussed teachers‟ voices regarding the National Examination in her

research (2010). Two of her participants had different opinion regarding how the National Examination should be conducted. One of them thought the National Examination should be conducted the way it had been conducted meanwhile

25

another one thought the National Examination should be conducted by adjusting

to vocational high school students‟ competencies. However, both of them thought

the National Examination should be based on English for Specific Purposes. b. Criticism of the National Examination

The National Examination had been a controversial issue in education in

Indonesia. Harti writes “discussion or polemic on the National Examination

always happens every year, even all years in Indonesia until they become latent

national education discussion” (n.d.). The controversy is constructed by two big

sides: the supporting side (the government) and the opposing side (the critics). Retno Listyarti, general secretary of the Federation of Indonesian Teachers Associations (Federasi Serikat Guru Indonesia/FSGI), said educators had been

“protesting the system of the examination implementation for years” (Jakarta

Globe, n.d.). In 2006, “a group of teachers filed a lawsuit at the Central Jakarta District Court demanding the abolition of the national exams. The court granted

the request and the government appealed the verdict” (The Jakarta Post, 2014, December 30). “The Supreme Court rejected the government‟s appeal” in 2009 (The Jakarta Post). They said “the examinations could only be held if their

implementation was improved” (The Jakarta Post).

1) The National Examination to Increase Intelligence and Improve Education Quality

Jusuf Kalla, when serving as vice president of Indonesia in 2004-2009, stated that the National Examination is important to help students increase their intelligence. He believed that students can increase their intelligence if they study and students had the motivation to study if there is test; the abolition of the

26

examination would make Indonesian students, as he mentioned it, stupid (Dharma, 2012). Because this examination (which he believed would make the students smarter) was based on one standard, Kalla was sure that the examination would make all students in Indonesia equally smart (despite different infrastructure each school had) (Dharma). Believing in the magic this National Examination had, Kalla also believed that in order to make our education level was equal to Singapore or Malaysia, in five years, all the government must have done was improving the examination difficult level every year (Dharma).

Different from Kalla who believed that the National Examination would be able to increase students‟ intelligence, Dharma (2012) stated that, other than

poverty, hunger, student mobility, health, safety, parent education, etc., students‟

intelligence would be affected by the input and process of education. Realistically, the input and process were still far from standard. In rural areas, many school buildings were broken; there were not enough teachers in schools, not enough books (Dharma). The National Examination was not a tool to make Indonesian students smarter. It was a tool to measure the output which was resulted from the input and process of education.

2) The National Examination Widen the Academic Achievement Gap

The government believed that the National Examination was an appropriate

way to improve students‟ motivation and achievement, even though, according to

critics, this believe the government had was just based on assumption (teKUN & eF, 2012). According to critics, this belief showed that the government neglected

other factors which might affect students‟ motivation, such as social and

27

socioeconomic status and school condition with academic achievement” (Willms,

2006; Fuchs & Womann, 2007, as paraphrased in teKUN & eF, 364).

As a part of universal interventions, the National Examination, which could be included in graduating test, could indeed improve academic achievement in one side (Willms, 2006 as quoted by teKUN & eF, 2012). However, on the other side,

it could “establish or even widen academic achievement difference based on

socioeconomic status” (Willms, as quoted by teKUN & eF).

As a high stake test, the National Examination also had many consequences. The positive consequence was that the examination would motivate the students to be the best (teKUN & eF, 2012). Other than that, it could also help authority to select graduates easily (teKUN & eF).

It also had negative consequences. Similar as its status as a part of universal interventions, the National Examination as a high stake test could also “widen

academic achievement gap based on socioeconomic status” (Dee & Jacob, 2006;

Willms, 2006 as paraphrased by teKUN & eF, 2012, 364). It only improved academic achievement of students with good social economical background or students who studied in schools with good condition (Dee & Jacob, 2006, as paraphrased by (teKUN & eF). If this type of examination is conducted since the students were in their young age, the gap between the students will be more drastic (teKUN & eF). This gap would form education elites (teKUN & eF).

High stake test, such as the National Examination, which was scientifically proven made for those with good socioeconomic background can also be seen

from Sarasvati‟s story in helping her students passing the National Examination

28

National Examination. According to Saravasti‟s explanation, her students did not

pass the National Examination because the school was not in standard condition and her students were from lower class families. Her students also could not pass the remedial examination and in order to join another remedial examination they had to pay 250.000 rupiahs or had access to lobby the authority. Some of her students did not pass the second remedial examination, even though one of them had already got in to a vocational school and did well in the school. In order to help this student passing the National Examination, Sarasvati had to lobby the

ministry of education. Sarasvati‟s conclusion on her experience was, “without

money, power, and networks, it would be almost impossible for a student who fails the examination to continue their schooling” (Sarasvati, 244).

3) An Incomprehensive Test

Even though as a high stake test the National Examination was considered able

to improve students‟ academic achievement, the improvement meant was only

limited to the tested subjects (Bishop, Mane, Bishop, & Moriarty, 2001; Phelp, 2001 as summarized in teKUN & eF, 2012). The subjects which were not tested tended to be neglected thus the consequence was that the curriculum became narrowed (Gayler, Chudowsky, Hamilton, Kober, & Yeager, 2004; Jones, Jones, & Hargrove, 2003; Watanabe, 2006 as paraphrased in teKUN & eF).

Other than that, student-centered learning process and learning process which focuses on creativity were also neglected because this type of examination required lots of drilling (Abrams, Pedulla, & Madaus, 2003; Jones, Jones, & Hargrove, 2003, Vogler, 2005; Zao, 2006 as paraphrased in teKUN & eF). Indonesian critics also argued that the National Examination does not cover three

29

aspects of objectives which include cognitive, affective, and psychomotor aspects (Harti, n.d). It only covers cognitive aspect (Harti).

4) The National Examination Had the Potential to Increase Dropping Out Risk

High stake test such as the National Examination has the potential to increase dropping out risk, especially for students from lower class (Cunningham & Sanzo, 2002; Dee & Jacob, 2006; Marchant & Paulson, 2005; National Research Council, 1997; Reardon, 1996; Warren, Jenkins & Kulick, 2006 as paraphrased in teKUN & eF, 2012), students from minority group (Dee & Jacob, 2006; National Research Council, 1997; Reardon, 1996; Warren, Jenkins, & Kulick, 2006 as paraphrased in teKUN & eF), and students with low academic achievement (Archer & Dresden, 1987; Bishop & Mane, 2001; Jacob, 2001 as paraphrased in teKUN & eF).

Sarasvati‟s research also showed how the National Examination as a high stake

test made two of her students dropped out of school and one of her students

almost dropped out of school (2012). Sarasvati‟s students were from lower class

family. Two students who dropped out did not join the second remedial examination. One of them was asked to work by his parents and another one was

too depressed to go on. One of Sarasvati‟s students who almost dropped out had

actually had entered a vocational school. Even though she finally joined the second remedial examination, she failed. The vocational school told Sarasvati they could not let her student continue her study if she did not pass. After lobbying members of ministry of education, her student was given the chance to do another exam which she could pass at last. According to Sarasvati, the students

30

finally dropped out of school because the National Examination demotivated the students by judging them with results showing if they pass or fail. The students also dropped out because the National Examination procedure, especially for those who failed the examination, was not clear. Another reason was because the input and process in her school was far from proper.

5) Pressure on Students and Teachers

High stake test can also give pressure on students (Gregory & Clarke, 2003 as paraphrased in teKUN & eF, 2012). The National Examination itself had reportedly causes depression and pressure on students (The Jakarta Post, 2014, December 30; Rachman, 2015, April 13; Jakarta Globe, n.d.). Students were also reported falling ill because they were too depressed in preparing for the examination Jakarta Globe.

Sarasvati (2012) showed how the National Examination made her students demotivated. Some of her students had to take two remedial examinations after failing the National Examination. Two of her students decided to drop out of school instead of joining the second remedial test. One of them decided not to join the second remedial test because she was depressed and pessimistic When studying for the second remedial test, the students joining the examination were also exhausted thus had very low motivation.

Other than making students demotivated, Sarasvati (2012) also stated that the National Examination put students, especially those who did not pass the examination, in uncertain position. Those who failed the National Examination were not told what was supposed to be done when they did not pass the examination: Should they have repeated their study in certain education level?

31

Should they have redone the examination in certain subjects, in all subjects? Should they have just stayed home and wait for the next examination or join kejar paket program (non-formal education)? (Sarasvati).

The National Examination also made it difficult for those who did not pass to continue their education (Sarasvati, 2012). As stated previously, the unclear procedure which the students should have followed when failing made only those having money, power, and network could get through the remedial examinations easily. It made those with no money, power, and network dropped out easily when faced with the fact that they failed the examination. If students could pass, they would not get the certificate. If they did not get the certificate, they could not continue their study.

The government was negligent to the students who were or might have been

depressed for the National Examination. “It is better to make a couple of students

stressful than a million of students stupid,” Kalla once said (Dharma, 2012, 86).

Not only to students, the National Examination was also reportedly brought pressure to teachers. In her writing Sarasvati told how stressful the teachers were in helping students be prepared for the remedial examination (2012). Indeed, research reports had shown that high stake test (including the National Examination) could give teachers pressure (Abrams, Pedulla, & Madaus, 2003, as paraphrased in teKUN & eF, 2012).

6) Cheating

Cheating practices had gone on for years during the National Examination. “In 2011, the FSGI received 102 reports of cheating, which increased to 317 in 2012

32

Post, 2015, April 15). Some students were reportedly bought answer keys and smuggled them to the examination rooms (Jakarta Globe, n.d.; Rachman, 2015, April 13). Some were also reportedly cheated by using mobile phones (Jakarta Globe).

Not only students, teachers and the government were also involved in cheating practices. Some teachers were accused of giving away answers (Rachman, 2015, April 13). Many schools in a rayon formed a National Examination Success Team, one of which tasks was to make sure that cross supervisions were not carried out (Dharma, 2012). When the case was reported to Ministry of Education, the institution only called the headmasters (Dharma). The inspector teacher was replaced. There was no investigation or sanction (Dharma). Dharma stated that the cheating practices had happened repeatedly and systematically in school, city to province level. However, there had never been any actions taken to stop or prevent the practices.

According to Dharma, the cheating practices were mainly engendered by the different input (teacher, facility, infrastructure, brain ware, software, hardware) given to each area in Indonesia (2012). Because the input was not equal, the quality was also not equal. Areas with low input and quality level chose to carry out cheating practice because if they do not, there was no way they would make even just more than 50% of the students pass the examination. In Balikpapan, a capital city in Kalimantan, more than 60% of students failed in the National Examination tryout. Many schools had 100% of the students failed the tryout. One vocational school which was prepared to have international standard could even only had 50% of the students successful in the tryout. The local governments did

33

not want to take the huge risk they should bear by letting so many students failed the examination. Cheating was the only way to boost high level of graduation percentage.

Sarasvati story about her experience in helping her students graduating also

support Dharma‟s analysis (2012). From Sarasvati story we can see why she

finally let her student cheating in the third remedial examination. After one national examination and two remedial examinations Sarasvati finally realized there was no way her students, who went to very poor school in a remote area, would pass the examination without cheating.

7) On Mapping Education Quality

The Law of the Ministry of Education Number 34 Year 2007 mentioned one of the National Examination functions as the tool to map the quality of students and schools and/or educational program. Sarasvati stated that, indeed, the map of

education quality “could be very useful as an input to design a better educational

system” and to improve education quality (2012, 231). However, Sarasvati

thought the National Examination alone was not enough to run this function because the National Examination only showed quantitative data of education

quality map. “Further qualitative research to find out why there are low quality

schools is needed” (Sarasvati, 231).

Dharma was pessimistic of the use of National Examination validity to map education quality. It is because cheating practices had always happened during National Examinations (Dharma).

34

c. Recommendation for a Better Assessment System

The following description is ideas proposed by teachers and critics regarding a better assessment system. The ideas include the importance of evaluating the National Examination, the importance of evaluating the government, what a better

assessment system is, and alternative option on how to determine students‟

graduation.

1) The Importance of Evaluating the National Examination

Previous elaboration on criticism towards the National Examination shows how weak government assumptions were regarding the National Examination. It also shows that the National Examination had more negative impacts compared to the positive.

It was thus important to evaluate the National Examination fairly. Those who supported and refused the National Examination needed to talk, test each of their assumptions regarding the National Examination, and find strategic solutions for better education for all Indonesian children. What were found needs to be shared to public (teKUN & eF, 2012).

2) The Importance of Evaluating the Government

Brown (1990, 5) as quoted in Sarasvati (2012, 227) stated that “assessment, as part of education, must be about promoting learning and opportunities, rather than

about sorting people into social roles.” This quotation and Sarasvati‟s experience

in helping her students face the National Examination brought Sarasvati to a

conclusion that it is important for the government to conduct “better educational

185

Jakarta Post, 2014, December 30), consideration for entrance selection of the next education level, and consideration for training and fund distribution to education units as the attempt to raise education quality (Menteri Pendidikan dan Kebudayaan Republik Indonesia [Mendikbud RI], 2015, p. 9).

Since it was no longer used to determine students‟ graduation, there was “no longer a minimum passing grade” (Nizam, head of the Ministry‟s Education Assessment Center, as quoted in Rachman, 2015, April 13). However, there was still “a minimum standard score for each subject”: 5.5 for each subject (Rachman). Those whose scores were below average were given the chance to retake the examination (Rachman).

FSGI stated the number of stressed-out students was less than last year. The students were still motivated in preparing for the examination because the results were still considered as one of the selection requirement of favorite universities. “It did not decrease the students‟ motivation. ... It decreased their burden,” Listartyi, said (Saut, P. D., 2015, April 13). Anies promised that the concept of the new version of the National Examination would be implemented perfectly. (The Jakarta Post, 2014, December 30).

b) Computer-based Examination for Certain Schools

In 2015, the National Examination was conducted online in 585 schools. Only few provinces have been prepared for it. The high schools involved in the online examination were more or less 2% of all high schools in Indonesia (The Jakarta Post, 2014, December 30; “UN Berbasis Komputer”, 2015, April 11).

186

According to Nizam, the computer-based national examination would be able to “cut down on costs to print and distribute the test” (Rachman, 2013, April 15). It can also anticipate the possibility of examination material leaks (“UN Berbasis Komputer”, 2015, April 11). Nizam further said that this type of examination would help the students to “enhance students‟ computer literacy skills, which he called a „basic need now‟” (Rachman).

Even though the computer-based national examination brought several advantages as mentioned above, its implementation was not simple. There were schools reporting technical obstacles in the process. In Jayapura, Papua, computer shortage was found in SMK 3 (Vocational High School 3) and five other schools. In SMK 3 there were only 160 computers for 480 students. Due to the condition, 420 students in the school “were unable to do the test simultaneously”. They “must wait in line: were divided into three groups, 160 students in each group” (Harahap & Somba, 2015, April 14).

A couple of other schools were faced with the fear that there would be power outage during the D-day of the examination. This fear made sense because in SMKN 1 Kupang power outage often happened (Dan, 2015, April 10). Because of the same fear, schools increased the electricity power from 23 thousand watt to 55 thousand watt (Dan). In SMAN 1 Cisarua, the only school in the city in which computer-based examination was conducted, technicians of PLN (Perusahaan Listrik Negara/State Electricity Company) and generator were prepared to prevent blackout during the examination (“UN Berbasis Komputer”, 2015, April 11). In Jayapura, Papua, one school “momentarily experienced a blackout during the last

187

five minutes of the examination, which caused all the computers to shut down.” (Harahap, R. & Somba N. D., 2015, April 14).

Internet connection was another factor which brought obstacles to the implementation of computer-based examination. In Jayapura, the examination committee found difficulties in downloading questions. They were supposed to be downloaded at 7 a.m. but the internet connection was only available at 8.30 a.m. (Harahap, R. & Somba N. D., 2015, April 14). In SMK 1 Tinambung, Majene regency, West Sulawesi, the examination was delayed up to 1 p.m due to the disrupted internet connection. Since the examination was divided in three sections, it was finished at 6 p.m. (Harahap, R. & Somba N. D.). In SMA Cendana and SMAN 8, Pekanbaru, Riau, the internet connection was disrupted for a moment therefore the students were not able to type their answers online (Harahap, R. & Somba N. D.). Riau Education Office Head, Dwi Agus Sumarno, stated that “the trouble was not due to the unpreparedness of schools in implementing the online exam” (Harahap, R. & Somba N. D.). He said the trouble was caused by the overburdened server at the Education Assessment Center. (Harahap, R. & Somba N. D.).

c. Cheating and Leaks

Even though the results of the 2015 National Examination were no longer used as the factor to determine student graduations, the national examination answer cheating and leaking cases were still found in many areas in Indonesia (Jong, 2015, April 16; Tarigan, 2015, April 16). The number of reports decreased (from 304 in 2014 to 91 in 2015 for cheating cases and from 11 in 2014 to 2 in 2015 for

188

answer key leaking cases) but there were still reports (The Jakarta Post, 2015, April 15).

FSGI reported answer key selling practices in East Java and DKI Jakarta. In Mojokerto, Lamongan, East Java, answer key was sold 14 million rupiahs. Each student pitched 50,000 rupiahs for it. In Jakarta, the answer key price was 14 million to 21 million rupiahs and each student pitched 50,000 to 100,000 to have it (The Jakarta Post, 2015, April 15).

They also reported practices of cheating by cell phone and small sheets of papers in Bekasi, Bogor, Bandung, Lamongan, and Jakarta (Tarigan, 2015, April 16). Cheating cases which involved The National Examination Success Team and Regional Education Service were also reported by FSGI.

In 2015, the national examination material was leaked. It had never happened before. Answer keys were usually leaked but in 2015 the material was. It was “reportedly available online for three days, from Saturday to Monday, before being deleted by Google after a request from the ministry” (Jong & Sundaryani, 2015, April 17). The material was distributed in Aceh, Yogyakarta, East Java, and Jakarta (Sundaryani, 2015, April 20; The Jakarta Post, 2015, April 15). There were 30 packages of material leaked (0.25 percent from the total 11,730 packages) (The Jakarta Post, 2015, April 15).

The government immediately responded to the case. They demanded students who suspected using the leaked material in answering the examination or schools which were suspected using them during the examination to redo the examination. According to Culture and Elementary and Secondary Education Minister, Anies

189

Baswedan, the students cheating could be analyzed by using the national examination results (Jong & Sundaryani, 2015, April 17). He said the results could detect systematic cheating because they had integrity index which measured honesty in the examination (Jong & Sundaryani; Sundaryani, 2015, April 20). Listyarti considered the government reaction to redo the examination unnecessary. She blamed the education policy for the leaks. “Just use this case as a lesson to show that education policy is mostly half-baked,” she said (Jong & Sundaryani).

Other than demanding to redo the examination, the government also planned to drag the parties responsible for the material multiplying to court. (Jong & Sundaryani, 2015, April 17). Police had been instructed to investigate the case. “Firms hired to print the examination materials would foot the bill,” Baswedan said (Jong & Sundaryani).

The government also instructed material destroying/sealing after the examination was over. Nizam said it was an international standard. “All 35 million exam papers would be destroyed after all junior high school, high school and vocational school examinations had been completed,” Baswedan said (Sundaryani, 2015, April 20). Even though the policy had been in place for years, “it had not been strictly enforced” (Sundaryani). Prosecution was what awaited the parties abusing the policy. “The material of the National Examination was state secret document,” Baswedan said (Sundaryani). Only parties authorized by the ministry had the right to store the material. Nizam said it was important to train teachers to “dispose of old exam materials properly,” (Sundaryani). Listyarti considered the policy „long-overdue‟ (Sundaryani). According to her, teachers

190

kept the materials because they found it necessary to to help students study for the next examinations (Sundaryani).