English testing and learning at the vocational school : student`s lived experience.
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
ABSTRACT
Neny Mariani, 2009. English Testing and Learning at the Vocational School:
Student’s Lived Experience. Yogyakarta: Sanata Dharma University, the
Graduate Program in English Studies
In the world of English education, in both theory and practice, testing which is
known as a means to measure learners’ abilities, continually poses big problems.
In the national level, testing has also become the main issue of a continual debate
and never-ending polemic. Despite all the shortcomings it may bring with it,
testing is widely used since it serves enormous practical functions. In this
situation, there is a need to promote learner-centered learning in that learning and
learner’s factors should be taken into consideration in the use and design of
English tests. This enhancement will accelerate learning autonomy; improve
quality and sustainability of learning process and achievement.
This research aimed at describing, interpreting and explaining how the vocational
school students experience English testing, revealing what testing means to them
and what in their perspective constitutes an ideal English test. As progressive
qualitative research, it was not meant to explain, predict, and control the student
and teacher properties and behavior. Instead, it was to reveal, describe and
understand better how the vocational school students experience English testing.
In order not to marginalize the participants as a result of the inquiry, this research
adopted the advocacy/participatory approach by starting with the suppression and
oppression issues and assumed that as the researcher I would proceed
collaboratively. Data were taken from an Indonesian vocational school with three
students (primary) and two English teachers (secondary) as data acquisition
sources. Data collection instruments included observation, interview and
document reviews. Verbal data, the participants’ narratives, were analyzed with
content analysis comprising descriptive and interpretative. To improve
trustworthiness of the research, data and methodological triangulation and thick
description were implemented over the course of the study. As a researcher, I
took an active participation and intervened by designing an on-going evaluation
in order to advocate becoming more empowered, autonomous, and self-fulfilling.
The research reveals that despite the similar concerns among the student
participants in testing, learning and achievement, there were unique emotional
experience and learning experience. It also reveals emergent themes around the
time of English testing which, in addition to the prefigured themes test backwash,
qualities, test content, learning and autonomy, includes status of test, fairness,
time allocation, achievement, competitiveness, preparedness. In addition, my
interpretation of what testing meant to the students as well as teachers shows that
there was a shift of concerns among teacher participants in different types within
xiv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
different contexts of testing. In the context of classroom testing, the teacher
participants had different concerns from the student participants, validity and
reliability, while the students were concerned about learning and achievement. In
the context of high-stakes testing, however, teachers and students shared similar
concerns, learning and achievement. That classroom testing potentially created a
point of clash and hurt student learning as its implication was due to the different
concerns in testing between the teachers and students. That high-stakes testing
created a gentleman’s agreement was due to the similar concerns. These research
results will lead to better understanding for the participants, students and
teachers, researcher as well as the audience. The participant improved awareness
of testing and learning and the researcher and audience improved emphatic
understanding of the participant experience, would consequently help facilitate
learner’s learning need and in turn lead to healthy learning process and better
results such as in particular to design a more progressive test and make a wiser
decision on why, when, how to use an educative test as assessment.
xv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
INTISARI
Neny Mariani, 2009. English Testing and Learning at the Vocational School:
Student’s Lived Experience. Yogyakarta: Universitas Sanata Dharma,
Kajian Bahasa Inggris
Di dalam pengajaran bahasa Inggris, baik dalam teori maupun praktek, tes yang
dikenal sebagai alat ukur kemampuan siswa masih menghadapi berbagai
masalah. Di kancah nasional, tes juga menjadi perdebatan dan polemik yang tak
kunjung selesai. Dengan segala kekurangannya, tes masih digunakan secara luas
karena masih banyak fungsinya. Dalam situasi seperti ini, diperlukan upaya
peningkatan pembelajaran yang berpusat pada pemelajar dalam hal faktor-faktor
kegiatan belajar dan siswa perlu diperhitungkan dalam penyusunan dan
pemakaian tes. Penekanan ini akan membawa dampak percepatan pembentukan
kemandirian belajar, meningkatkan kualitas dan keberlangsungan proses serta
keberhasilan proses dan prestasi belajar.
Penelitian ini mencoba menggambarkan, memahami, dan menjelaskan
bagaimana pengalaman siswa SMK dalam tes Bahasa Inggris dan mengungkap
apa makna tes bagi mereka serta apa pandangannya yang merupakan tes ideal itu.
Karena bersifat kualitatif, penelitian ini tidak dimaksudkan untuk menerangkan,
memprakirakan dan mengendalikan unsur-unsur yang ada pada siswa dan guru,
termasuk perilakunya. Namun, penelitian ini akan membuka tabir,
menggambarkan dan memahami secara lebih baik bagaimana pengalaman siswa
SMK terhadap tes Bahasa Inggris.
Untuk tidak mengesampingkan peserta sebagai akibat dari penelitian ini,
pendekatan advocacy/participatory digunakan dengan dimulai adanya isu
penekanan dan penindasan, dan anggapan bahwa peneliti akan melakukan
tugasnya dengan kompromi. Data diperoleh dari sebuah SMK di Indonesia
dengan sumber data dari tiga siswa (data primer) dan dua guru Bahasa Inggris
(data sekunder). Alat pengumpulan data berupa observasi, wawancara, dan telaah
dokumen. Data verbal yang berupa ucapan peserta dianalisis dengan analisis isi
yang meliputi analisis deskriptif and interpretatif. Untuk meningkatkan
keterpercayaan hasil penelitian, triangulasi data dan metodologi serta thick
description dilakukan selama proses dan pelaporan hasil penelitian. Sebagai
peneliti, saya terlibat aktif mengambil bagian dan melakukan intervensi dengan
cara melakukan refleksi kegiatan yang terus menerus dalam membantu
menyuarakan agar lebih berdaya, mandiri dan dapat memenuhi kebutuhan
belajarnya sendiri.
Penelitian menunjukkan bahwa meskipun di antara sesama siswa terdapat
persamaan dalam kepentingan yaitu belajar dan prestasi, mereka memiliki
perbedaan dalam pengalaman emosional dan pengalaman belajar. Penelitian ini
juga menunjukkan munculnya tema-tema baru seputar pelaksanaan tes yaitu
xvi
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
selain dampak, kualitas, isi tes, belajar dan kemandirian, juga mencakup status
tes, keadilan, alokasi waktu, prestasi, kebersaingan, dan kesiapan. Sebagai
tambahan, interpretasi saya mengenai apa makna tes bagi siswa dan guru SMK
menunjukkan adanya pergeseran kepentingan guru bila terdapat perbedaan jenis
dan konteks pengujian. Dalam konteks tes berbasis kelas, guru mempunyai
perbedaan kepentingan dengan siswa, yaitu validitas dan reliabilitas; sementara
itu siswa mempunyai kepentingan belajar dan prestasi. Namun dalam konteks
tes skala besar, guru dan siswa mempunyai kepentingan yang serupa, yaitu
belajar dan prestasi. Bahwa tes berbasis kelas berpotensi menimbulkan konflik
dan menghambat proses belajar siswa sebagai implikasinya itu terkait adanya
perbedaan kepentingan dalam tes antara guru dan siswa tersebut. Tes berskala
besar menimbulkan adanya saling pengertian itu karena kesamaan kepentingan.
Hasil penelitian ini akan memberikan pemahaman yang lebih baik bagi peserta,
siswa dan guru, peneliti dan khalayak. Meningkatnya kesadaran diri peserta
terhadap pengujian dan pemelajaran, dan rasa empati dari peneliti dan khalayak
akan membawa ke arah proses belajar yang sehat dan hasil yang lebih baik, dan
membantu mengakomodasi kepentingan dan kebutuhan belajar siswa, misalnya
dengan merancang tes yang bersifat progresif serta melakukan kebijakan tentang
mengapa, kapan, bagaimana menggunakan tes sebagai alat penilaian yang
mendidik.
xvii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
ENGLISH TESTING AND LEARNING
AT THE VOCATIONAL SCHOOL:
STUDENT’S LIVED EXPERIENCE
A THESIS
Presented in Partial Fulfillment of the Requirements
for the Master’s Degree
in English Language Studies
by
Neny Mariani
Student Number: 056332027
THE GRADUATE PROGRAM IN ENGLISH STUDIES
SANATA DHARMA UNIVERSITY YOGYAKARTA
2009
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
i
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
ii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
STATEMENT OF ORIGINALITY
This is to certify the originality of the written study. No single part of this thesis
but materials cited as references, either explicitly or implicitly, shows similarities or
resemblance with any other work, and therefore to the best of my knowledge, no
copyright violation is indicated. Hereby I declare that this thesis represents my
own work. I assume full responsibility for the content of the work and bear all
consequences it may result in.
Yogyakarta,
21st December 2009
Neny Mariani
iii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
LEMBAR PERNYATAAN
PERSETUJUAN PUBLIKASI KARYA ILMIAH
UNTUK KEPENTINGAN AKADEMIS
Yang bertanda tangan di bawah ini, saya mahasiswa Universitas Sanata Dharma
Yogyakarta dengan
Nama
: Neny Mariani
Nomor mahasiswa: 056332027
menyatakan bahwa demi pengembangan ilmu pengetahuan memberikan kepada
Perpustakaan Universitas Sanata Dharma Yogyakarta karya ilmiah yang berjudul:
“English Testing and Learning at the Vocational School: Student’s Lived Experience”
yang saya tulis beserta perangkat terkait yang tersedia.
Dengan demikian kepada Perpustakaan Universitas Sanata Dharma Yogyakarta
saya berikan hak dan wewenang untuk menyimpan, mengalihkan dalam bentuk media
lain, mengolahnya dalam bentuk pangkalan data, mendistribusikan secara terbatas, dan
mempublikasikannya di internet atau media lain untuk kepentingan akademis tanpa perlu
meminta ijin dari saya maupun memberikan royalti kepada saya selama tetap
mencantumkan nama saya sebagai penulis.
Demikian pernyataan ini saya buat dengan sesungguhnya.
Dibuat di
: Yogyakarta
Pada tanggal: 21 Desember 2009
Yang menyatakan,
Neny Mariani
iv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
ACKNOWLEDGMENTS
First of all, I wish to offer my biggest adoration to the Almighty, who has
granted me an opportunity to pursue further education and kept sending so many
wonderful people I learn from and work with. I am so grateful for all the blessings,
inspiration and never-ending guide from nothing to something, prior to, during, and
upon completion of this master’s thesis. I guess there will never be bigger celebration to
mark the accomplishment of this project than expressing the words of glory, "Praise be
to Allah, the Cherisher, Sustainer of the worlds!". Also, I do not want to miss this
opportunity to express a word of honor to my parents, from whom I received my first
education in my earliest years, for raising me up and introducing the Most Gracious and
Merciful.
Although the content of this thesis is essentially and eventually my own
responsibility, this work by all means results from a great number of contributors.
Therefore I wish to take this opportunity to acknowledge and appreciate all kinds of
contribution which I have received directly and indirectly from those whose names I will
not be able to print out here one by one.
First of all, I would extend my particular gratitude to my adviser, Pak Bis (Dr J
Bismoko), for his great dedication and allowing me to get an easy access for correction
and consultation. It would almost be impossible to do this research project without the
existence of such a prominent figure behind the preparation, implementation and
presentation of the work.
I have been largely indebted to all the lecturers and everyone in the graduate
v
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
program of this university. I would like to thank the Graduate Program Director, Prof.
Dr. Augustinus Supratiknya for legalizing this report. I thank the Board of Examiners,
Pak Muk (FX. Mukarto, Ph.D), Pak Dwi (Dr. B.B. Dwijatmoko, M.A), Pak Alip (Dr. Fr.
B. Alip, M.Pd, M.A.) for helping me finalize this work with the thorough examination. I
also thank the rest of the lecturers, Pak Barli (Drs. Barli Bram M.S.), Bu Novi (Dr.
Novita Dewi, M.S., M.A. (Hons)), and Pak Pomo (Prof. Dr. Soepomo Poedjosoedarmo)
for their "shower of sparks" and "drops of dews".
For the financial and non financial support as well as attention, my gratitude goes
to LPMP Yogyakarta, including Bu Anna (Dra. Anna Maria Hendraswari, M.A.), Pemda
Kabupaten Kulon Progo (including Pak Ridwan) and Dinas Pendidikan Kabupaten
Kulon Progo (including Pak Yatiman ).
Next, I would express my sincere gratitude to my research participants, my
fellow English teachers and students, for every word they have contributed, their
honesty, and the time and effort they have spent to reply to my enquiries. For me, they
have been sunshine in the rainy day and water in the desert. I would like to express my
thankfulness to everybody in the school where I spend wonderful years, SMK Negeri 1
Pengasih, which includes the school principal Drs. H. Rumawal, my colleagues and
students, for the encouragement, challenge, spiritual, technical and financial support.
I thank all authors and researchers whose work is very valuable and contributive
to my project. I have made every effort to trace all the resources the best I could, and I
hope I do not miss a thing.
To all my fellow students and my acquaintances, I would say a big thank-you for
the share of knowledge and feelings. All the years when we were involved in such tough
vi
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
and fierce discussions, arguing for and against, sharing laughter and tears have been so
precious to me. I will always miss the time when the feeling of friendship was that
strong.
Upon completion of the work, I would like to dedicate this thesis to the
Motherland and to the world. I realize this work will be more meaningful it gains
attention from wider audience. I look forward to and welcome any response coming
from all directions.
Yogyakarta, 21st December 2009
Neny Mariani
vii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
TABLE OF CONTENTS
TITLE PAGE
APPROVAL PAGE
DEFENSE APPROVAL PAGE
STATEMENT OF ORIGINALITY
STATEMENT OF LICENCE AGREEMENT
ACKNOWLEDGEMENTS
TABLE OF CONTENTS
LIST OF FIGURES
LIST OF TABLES
LIST OF ABBREVIATIONS
ABSTRACT
INTISARI
CHAPTER I
A.
B.
C.
D.
E.
F.
1
INTRODUCTION
BACKGROUND
PROBLEM IDENTIFICATION
PROBLEM LIMITATION
PROBLEM STATEMENT
RESEARCH GOALS
RESEARCH BENEFIT
CHAPTER II
i
ii
iii
iv
v
viii
xi
xii
xiii
xiv
xvi
1
5
10
11
12
12
LITERATURE REVIEW
14
A. THEORETICAL REVIEW
1. Assessment
2. Language testing, English testing and backwash effects
3. Learning and learning autonomy
4. Language learning and language learning autonomy
5. Vocational school
6. Experience and attitude
7. Qualitative research and methodologies
B. THEORETICAL FRAMEWORK
14
14
18
29
34
37
39
40
46
CHAPTER III
55
A.
B.
C.
D.
E.
METHODOLOGY
RESEARCH DESIGN
DATA SOURCES
TYPES OF DATA
DATA COLLECTION TECHNIQUES
DATA PROCESSING PROCEDURE
1. DATA COLLECTING
2. DATA ANALYZING
viii
55
60
62
63
66
66
67
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
F. TRUSTWORTHINESS OF RESEARCH RESULTS
CHAPTER IV
.
ANALYSIS RESULTS
A. DATA AND SOURCE SELECTION
B. INSTRUMENT VALIDATION AND ACCOUNTABILITY
C. DATA GATHERING, RECORDING AND PRESENTATION
D. DATA ANALYSIS ACTUALIZATION
E. DISCUSSION: DESCRIPTION AND INTERPRETATION
1. Student participant portraits in English testing and learning
2. Thematic description and illustrative quotations
3. Participants’ meaning of testing
129
CHAPTER V
CONCLUSIONS, IMPLICATIONS,
AND RECOMMENDATIONS
A. CONCLUSIONS
B. IMPLICATIONS
C. RECOMMENDATIONS
68
73
73
78
80
85
106
106
115
144
144
148
149
BIBLIOGRAPHY
151
APPENDICES
158
Appendix 1 Interview probes
1.1 Questions to students
1.2 Questions to teachers
Appendix 2 Coding blue print
2.1 Participant: students
2.2 Participant: teachers
2.3 Observation on test implementation
2.4 Inductive codes
Appendix 3 Observation data of events and behavior from official meetings
and environs
3.1 Official meeting 1 ( external)
3.2 Official meeting 2 ( external)
3.3 Official meeting 3 ( external)
3.4 Official meeting 4 (internal)
Appendix 4 Observation data of events and behavior from test sites
4.1 Classroom test 1
4.2 Classroom test 2
4.3 Classroom test 3
4.4 Classroom test 4
4.5 Classroom test 5
4.6 High-stakes test 1
4.7 High-stakes test 2
ix
158
158
158
159
159
160
161
161
162
162
163
165
165
166
166
166
167
167
167
168
169
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
4.8 High-stakes test 3
Appendix 5 Observation data of physical and non physical environs
5.1 Situation prior to classroom test 1
5.2 Situation prior to classroom test 2
5.3 Situation prior to high-stakes test 1
5.4 Situation prior to high-stakes test 2
5.5 Situation after high-stakes test 1
5.6 Situation after high-stakes test 2
5.7 Situation after high-stakes test 3
Appendix 6 Researcher’s diary
Appendix 7 Interview data
7.1 Student Atik 1
7.2 Student Atik 2
7.3 Student Siti 1
7.4 Student Siti 2
7.5 Student Endang
7.6 Teacher Iin 1
7.7 Teacher Iin 2
7.8 Teacher Nanik 1
7.9 Teacher Nanik 2
Appendix 8. News-stories and information
8.1 Radio news
8.2 Television news
8.3 Printed news
8.3.1 Printed news 1
8.3.2 Printed news 2
Appendix 9 Documents
9.1 Official document 1
9.2 Official document 2
9.3 Student Endang’s answer sheet
9.4 Student Atik’s work
9.5 Student Atik’s school report
9.6 Classroom test sheet
9.7 High-stakes test booklet
9.8 Test-result report
9.8.1 Score report from high-stakes test 1
9.8.2 Score report from high-stakes test 2
x
170
171
171
172
172
173
174
174
174
175
183
183
191
201
207
214
221
226
238
245
249
249
250
257
257
258
259
259
260
261
262
263
266
267
269
269
270
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
LIST OF FIGURES
1. Figure 1.1 Tripartite partnership model
2. Figure 1.2 First, essential stage in learning enhancement partnership
3. Figure 1.3 Learning and testing relationship
4. Figure 1.4 Testing and stakeholders
5. Figure 2.1 Communication in testing environment stage 1, 2, 3
6. Figure 2.2 Bachman’s categorization of the use of tests
7. Figure 2.3 Brown’s language test type categorization
8. Figure 2.4 McNamara’s language test type categorization
9. Figure 2.5 Test type categorization
10. Figure 2.6 Testing in the language program toward independent learning
11. Figure 2.7 Positioning English testing at the Vocational School
12. Figure 2.8 English testing aspects categorization
13. Figure 3.1 From research questions to research results
14. Figure 3.2 Types of data
15. Figure 3.3 Data processing procedure
16. Figure 4.1 Student’s lived experience in English testing and learning
at the vocational school seen from various angles
17. Figure 4.2 Data processing stages
18. Figure 4.3 High-stakes test and thematic description, gleaned from
secondary data sources
19. Figure 4.4 Different interests within testing
xi
6
7
8
9
21
25
27
27
29
47
50
52
59
62
66
79
87
118
140
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
LIST OF TABLES
Table 2.1 Theoretical concept map
Table 3.1 Research specification
Table 4.1 Student participant data
Table 4.2 Teacher participant data
Table 4.3 Observation blueprint
Table 4.4 Sample data observation 1
Table 4.5 Sample data observation 2
Table 4.6 Probing interview questions
Table 4.7 A priori coding inventory for student participants
Table 4.8 Sample corpus of data
Table 4.9 Sample tabulating table
Table 4.10 Semi-ordered list of responses to the open-ended question
“Bagaimana perasaanmu sebelum, pada saat, dan setelah
mengikuti ulangan harian?”
Table 4.11 Category of responses to the open-ended question
“Bagaimana perasaanmu sebelum, pada saat, dan
setelah mengikuti ulangan harian?”
Table 4.12 Emerging themes and categories from student participants’
narratives and behavior
Table 4.13 Components, categories and indicators constituting
environment around English testing implementation
Table 4.14 Emergent categories from primary and secondary data
xii
54
58
76
76
81
82
83
84
85
88
96
101
103
104
116
117
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
LIST OF ABBREVIATIONS
CARLA: Center for Advanced Research on Language Acquisition
Depdiknas: Departemen Pendidikan Nasional
EFL: English as a Foreign Language
ESL: English as a Second Language
ESP: English for Specific Purpose
FHI: Family Health International
LOTE: Language Other Than English
MTs: Madrasah Tsanawiyah
RRI: Radio Republik Indonesia
SLA: Second Language Acquisition
SMK: Sekolah Menengah Kejuruan
SMP: Sekolah Menengah Pertama
TOEIC: Test of English for International Communication
TVRI: Televisi Republik Indonesia
UWE: University of the West England
xiii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
ABSTRACT
Neny Mariani, 2009. English Testing and Learning at the Vocational School:
Student’s Lived Experience. Yogyakarta: Sanata Dharma University, the
Graduate Program in English Studies
In the world of English education, in both theory and practice, testing which is
known as a means to measure learners’ abilities, continually poses big problems.
In the national level, testing has also become the main issue of a continual debate
and never-ending polemic. Despite all the shortcomings it may bring with it,
testing is widely used since it serves enormous practical functions. In this
situation, there is a need to promote learner-centered learning in that learning and
learner’s factors should be taken into consideration in the use and design of
English tests. This enhancement will accelerate learning autonomy; improve
quality and sustainability of learning process and achievement.
This research aimed at describing, interpreting and explaining how the vocational
school students experience English testing, revealing what testing means to them
and what in their perspective constitutes an ideal English test. As progressive
qualitative research, it was not meant to explain, predict, and control the student
and teacher properties and behavior. Instead, it was to reveal, describe and
understand better how the vocational school students experience English testing.
In order not to marginalize the participants as a result of the inquiry, this research
adopted the advocacy/participatory approach by starting with the suppression and
oppression issues and assumed that as the researcher I would proceed
collaboratively. Data were taken from an Indonesian vocational school with three
students (primary) and two English teachers (secondary) as data acquisition
sources. Data collection instruments included observation, interview and
document reviews. Verbal data, the participants’ narratives, were analyzed with
content analysis comprising descriptive and interpretative. To improve
trustworthiness of the research, data and methodological triangulation and thick
description were implemented over the course of the study. As a researcher, I
took an active participation and intervened by designing an on-going evaluation
in order to advocate becoming more empowered, autonomous, and self-fulfilling.
The research reveals that despite the similar concerns among the student
participants in testing, learning and achievement, there were unique emotional
experience and learning experience. It also reveals emergent themes around the
time of English testing which, in addition to the prefigured themes test backwash,
qualities, test content, learning and autonomy, includes status of test, fairness,
time allocation, achievement, competitiveness, preparedness. In addition, my
interpretation of what testing meant to the students as well as teachers shows that
there was a shift of concerns among teacher participants in different types within
xiv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
different contexts of testing. In the context of classroom testing, the teacher
participants had different concerns from the student participants, validity and
reliability, while the students were concerned about learning and achievement. In
the context of high-stakes testing, however, teachers and students shared similar
concerns, learning and achievement. That classroom testing potentially created a
point of clash and hurt student learning as its implication was due to the different
concerns in testing between the teachers and students. That high-stakes testing
created a gentleman’s agreement was due to the similar concerns. These research
results will lead to better understanding for the participants, students and
teachers, researcher as well as the audience. The participant improved awareness
of testing and learning and the researcher and audience improved emphatic
understanding of the participant experience, would consequently help facilitate
learner’s learning need and in turn lead to healthy learning process and better
results such as in particular to design a more progressive test and make a wiser
decision on why, when, how to use an educative test as assessment.
xv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
INTISARI
Neny Mariani, 2009. English Testing and Learning at the Vocational School:
Student’s Lived Experience. Yogyakarta: Universitas Sanata Dharma,
Kajian Bahasa Inggris
Di dalam pengajaran bahasa Inggris, baik dalam teori maupun praktek, tes yang
dikenal sebagai alat ukur kemampuan siswa masih menghadapi berbagai
masalah. Di kancah nasional, tes juga menjadi perdebatan dan polemik yang tak
kunjung selesai. Dengan segala kekurangannya, tes masih digunakan secara luas
karena masih banyak fungsinya. Dalam situasi seperti ini, diperlukan upaya
peningkatan pembelajaran yang berpusat pada pemelajar dalam hal faktor-faktor
kegiatan belajar dan siswa perlu diperhitungkan dalam penyusunan dan
pemakaian tes. Penekanan ini akan membawa dampak percepatan pembentukan
kemandirian belajar, meningkatkan kualitas dan keberlangsungan proses serta
keberhasilan proses dan prestasi belajar.
Penelitian ini mencoba menggambarkan, memahami, dan menjelaskan
bagaimana pengalaman siswa SMK dalam tes Bahasa Inggris dan mengungkap
apa makna tes bagi mereka serta apa pandangannya yang merupakan tes ideal itu.
Karena bersifat kualitatif, penelitian ini tidak dimaksudkan untuk menerangkan,
memprakirakan dan mengendalikan unsur-unsur yang ada pada siswa dan guru,
termasuk perilakunya. Namun, penelitian ini akan membuka tabir,
menggambarkan dan memahami secara lebih baik bagaimana pengalaman siswa
SMK terhadap tes Bahasa Inggris.
Untuk tidak mengesampingkan peserta sebagai akibat dari penelitian ini,
pendekatan advocacy/participatory digunakan dengan dimulai adanya isu
penekanan dan penindasan, dan anggapan bahwa peneliti akan melakukan
tugasnya dengan kompromi. Data diperoleh dari sebuah SMK di Indonesia
dengan sumber data dari tiga siswa (data primer) dan dua guru Bahasa Inggris
(data sekunder). Alat pengumpulan data berupa observasi, wawancara, dan telaah
dokumen. Data verbal yang berupa ucapan peserta dianalisis dengan analisis isi
yang meliputi analisis deskriptif and interpretatif. Untuk meningkatkan
keterpercayaan hasil penelitian, triangulasi data dan metodologi serta thick
description dilakukan selama proses dan pelaporan hasil penelitian. Sebagai
peneliti, saya terlibat aktif mengambil bagian dan melakukan intervensi dengan
cara melakukan refleksi kegiatan yang terus menerus dalam membantu
menyuarakan agar lebih berdaya, mandiri dan dapat memenuhi kebutuhan
belajarnya sendiri.
Penelitian menunjukkan bahwa meskipun di antara sesama siswa terdapat
persamaan dalam kepentingan yaitu belajar dan prestasi, mereka memiliki
perbedaan dalam pengalaman emosional dan pengalaman belajar. Penelitian ini
juga menunjukkan munculnya tema-tema baru seputar pelaksanaan tes yaitu
xvi
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
selain dampak, kualitas, isi tes, belajar dan kemandirian, juga mencakup status
tes, keadilan, alokasi waktu, prestasi, kebersaingan, dan kesiapan. Sebagai
tambahan, interpretasi saya mengenai apa makna tes bagi siswa dan guru SMK
menunjukkan adanya pergeseran kepentingan guru bila terdapat perbedaan jenis
dan konteks pengujian. Dalam konteks tes berbasis kelas, guru mempunyai
perbedaan kepentingan dengan siswa, yaitu validitas dan reliabilitas; sementara
itu siswa mempunyai kepentingan belajar dan prestasi. Namun dalam konteks
tes skala besar, guru dan siswa mempunyai kepentingan yang serupa, yaitu
belajar dan prestasi. Bahwa tes berbasis kelas berpotensi menimbulkan konflik
dan menghambat proses belajar siswa sebagai implikasinya itu terkait adanya
perbedaan kepentingan dalam tes antara guru dan siswa tersebut. Tes berskala
besar menimbulkan adanya saling pengertian itu karena kesamaan kepentingan.
Hasil penelitian ini akan memberikan pemahaman yang lebih baik bagi peserta,
siswa dan guru, peneliti dan khalayak. Meningkatnya kesadaran diri peserta
terhadap pengujian dan pemelajaran, dan rasa empati dari peneliti dan khalayak
akan membawa ke arah proses belajar yang sehat dan hasil yang lebih baik, dan
membantu mengakomodasi kepentingan dan kebutuhan belajar siswa, misalnya
dengan merancang tes yang bersifat progresif serta melakukan kebijakan tentang
mengapa, kapan, bagaimana menggunakan tes sebagai alat penilaian yang
mendidik.
xvii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
CHAPTER I
INTRODUCTION
To identify and expose the main research problems elaborately, in this
introductory chapter I will examine concepts and issues in English education,
specifically English testing in the global as well as local context. I will begin with the
background of the study and consecutively discuss the problem identification, problem
limitation, problem statement, research goals, and research benefits.
A. BACKGROUND
Over the last few decades, as writers say, language test theory has put a great
deal of emphasis on three major areas. First is the nature of language; that is when we
talk about skills and language components. Next is the nature of language use; that is
when we talk about authenticity. Third is the nature of measurement; that is when we
talk about validity and reliability. This can be seen from the work of, take for example,
Bachman (1995) and McNamara (1996). It is not surprising then that in practice test
developers and specialists are reportedly deeply concerned with anything related to
defining language competence, understanding the factors which influence performance
on language tests and finding valid and reliable ways of measuring language competence
and development, and recently greatly worried about whether test materials and tasks
represent actual use of language for communication. In conclusion, we are excessively
concerned with the overt function of the test, as a means of measurement and
assessment, ignoring the covert but influential function, that is, as a learning experience.
1
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
It is clear for us that most test theories as proposed by Bachman (1995),
McNamara (1996) and Brown (2005) imply that they place test-takers as the objects
of the test. Test-takers are considered as “something we can do measurement upon”.
It is not clear whether it will give any benefits or harms to them. The definitions
imply that test is designed for the sake of other parties, while the primary
stakeholders, which are in fact the test-takers themselves, are totally forgotten. This
is what I call oppression, suppression and colonization. Up to this point, it is
obvious that so far we have been busy dealing with constructing a test for its own
sake and for its own right, rather than catering for student learning needs or
facilitating learning in the stage of evaluation within the teaching and learning
process. Whether the test construction has harmful or positive effects on learning
remains out of discussion and little or even almost nothing about learning has been
taken into consideration in the design and use of language tests in most EFL/ESL
courses. The lack of discussion and the absence of such consideration in the test
design and use is greatly regrettable. In relation to this, Hughes’ (2003:1) assertion,
showing the inadequacy in the test design and use, is worth quoting, “[…] a great
deal of language testing is of very poor quality. Too often language tests have
harmful effects on teaching and learning […]”.
In this problematic situation, we then need a review on the most current
educational and learning issues. Life-long education has been widely noted since
early times and Delors et al. (1998:20) reaffirm it by iterating that the concept of
education pursued throughout life recently emerges as one of the keys to the twenty
first century. Education, an ongoing process of improving knowledge, skills and
attitude or behavior, with the four pillars on which education throughout life is
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
based, learning to know, learning to do, learning to be and learning to live together,
is said to have a fundamental role to play in personal and social development in that
it is an exceptional means of bringing about personal development and building
relationship among individuals, groups and nations.
Next, let us refer to the postmodernism theory, with its progressivism
philosophy, which says education should be directed to make learners empowered,
autonomous, and eventually become self-fulfilling individually and socially in their
own world. Finney (2002:73) puts it this way,
The purpose of education from the point of view of the process model is to
enable the individual to progress toward self-fulfillment. It is concerned with
the development of understanding, not just the passive reception of
‘knowledge’ or the acquisition of specific skills. The goals of education are
not defined in terms of particular ends or products, but in terms of the
processes and procedures by which the individual develops understanding
and awareness and creates possibilities for future learning. (Italicized words
are my emphasis)
Here we should note that the key word to education is “self-fulfillment”, and
consequently we can say we need learner-centeredness, an approach to education
that aims at developing in each student a sense of responsibility for his or her own
learning (The University’s Strategic Plan, 1995 as quoted in Hall, 2000:2). In other
words it means the development of learner autonomy should be the heart of any
educational program, which certainly includes language teaching and learning.
To ensure the enhancement of learner-centered learning and the development
of learner autonomy as stated above, therefore, I felt it urgent to do research on a
particular topic so that learning, which is the most essential of all in education,
would be appropriately ensured, adequately addressed and consistently become the
main consideration in the whole process of designing and implementing educational
course all the way from assessing learner’s learning needs to the stage of program
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
evaluation, including testing.
Test, which is in a general sense the main instrument in learner assessment,
is indispensable part of the ESL/EFL/ESP teaching and learning process, and it has
been my interest to do a research on. The choice of the topic was due to the
following reasons. First, as you observe, test as the main instrument of assessment
remains controversial and problematic in both theory and practice in that students’
learning needs and viewpoints in testing has been paid little attention. In addition, we
can refer to that of two prominent advocates of ESP Hutchinson and Waters
(1994:146), who realize and notice there is a shortage of sound theoretical or
empirical basis for ESP testing and lack of discussion or guidance on ESP testing:
“Far more research is therefore needed before we will really know what the
requirements of a good ESP test are”. Second, my personal and professional
background as an EFL teacher had driven me conduct numerous kinds of tests for
vocational school students in the local, regional as well as national level. Thus, in
Trochim’s words (2006:1), I had been living and grown with the phenomenon, or in
other words, it is my “direct experience of the phenomenon”. Therefore in relation to
the learning enhancement and the learner autonomy development as I have already
described, then I concentrated on the topic, English testing.
In relation to the topic, I believed, the focus of further research on testing
should be on learning factors and an understanding of the impacts of language
testing to learning, instead of language or test property analysis. The statement of
Chalhoub-Deville (2003:380) is worth quoting to confirm the idea,
Given current indications in language testing, the nature of the components
underlying the L2 construct as well as the manner in which these components
interact are issues that researchers need to address in the twenty-first century.
4
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
Indonesian vocational schools, whose mission is to provide a middle-class
workforce and to improve their competitiveness, are greatly concerned with their
EFL or more specifically ESP teaching and learning. A policy shift from a supplydriven system to a demand-driven system guided by labor market signal, curriculum
reform, developing syllabus design, improvement on learning resources, teacher
training, the adoption of the international standardized test, periodical debate contest
are clear illustrations of the struggle. Nevertheless, the major common problem
remains. Having spent their years of schooling, a number of students are still
considered ‘unsuccessful’ English learners, not only emerging with a very low
proficiency but also indicating low motivation and having a high dependence on
teachers in their learning.
Thus, despite tremendous progress in curriculum reforms and policies, two
significant things, which are implications of the curriculum reforms itself, remain
untouched: understanding of student’s learning needs and experience in testing. In
such a situation, to do research on English testing and learning is highly relevant
since the results will surely be contributive to learning enhancement
and
sustainability and eventually help overcome the persistent problems in education.
B. PROBLEM IDENTIFICATION
The problematic situation, which takes place in both theory and practice,
implies a kind of oppression or colonization upon learners, and then it raises
questions as “If a test is a measurement, why measure?, “Is measuring the ultimate
goal in all education?”, ”Where is in language testing the room for learning
facilitation which builds learner’s autonomy?”, “Is a traditional paper and pencil
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
test totally inauthentic in language testing?”, “Does something inauthentic have no
meaning and bring bad impacts to learning?”, “If a test is an instrument for us to
use, why is it not designed as user-friendly?”. With all these problematic questions,
it is clear that an educational reform is urgent to do, and further research on English
testing is the right way to implement the reform.
To get out of the problematic situation in the world of education, it is a
good idea to refer to what Delors et al. (1998: 27) state,
The main parties contributing to the success of educational reforms are,
first of all, the local community, including parents, school heads and
teachers; secondly, the public authorities; and thirdly, the international
community. Many past failures have been due to insufficient involvement
of one or more of these partners. (bold-faced is my emphasis)
From here, it is obvious that the involvement of all parties in the reform is
obligatory; local community’s contribution takes precedence over the others, and
that the absence of any party in the involvement will lead to another failure. Figure
1.1 presents a clear illustration of this.
First Party
Local Community
Educational Reforms
Second Party
Public Authority
Third Party
International Community
Figure 1.1 Tripartite partnership model in education reform
(visualizing the idea of Delors et al., 1998 )
6
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
Furthermore, for the success of educational reforms, Delors et al. (1998:27)
also necessitate a dialog among the parties as they point out,
Local community participation in assessing needs by means a dialogue with
the public authorities and group concerned in society is a first, essential stage
in broadening access to education and improving its quality. (my emphasis)
This is in line with learner-centered teaching orientation which begins with the
premise that the learner is the central reference point in decision making (Tudor,
1997). Thus, if we want to enhance learner-centered learning and to know the
requirements of a good test, learning-needs assessment is significant to do, and
having a dialog with test stakeholders to ensure sufficient involvement of pu
blic authority and local community is highly relevant. This can be illustrated in
Figure 1.2.
Local Community:
Students, parents, teachers, school heads
Educational Reform:
Enhancing learner-centered learning
Public Authority:
Government, researchers
Figure 1.2 First, essential stage in learning enhancement
partnership (developed from the idea of Delors et al.,
1998)
One of the logical consequential applications of this theory is that it is urgent
to do research which cares about the learners’ concern by having a dialog with
students and teachers as the local community party. Again, Tudor (1997) confirms
this idea, stating that needs analysis and goal-setting can be more effective if
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
students inform them with their knowledge and experience; and learning is more
successful if methodology and study mode take into account student preferences and
characteristics. In this research, therefore I as the researcher representing the public
authority party will facilitate what they want to say in order to implement the reform
effectively.
To begin with, let us study how learning and testing is interrelated.
Here, Figure 1.3 shows that in the context of educational program, testing can be
seen as a learning environment, which constitutes one of factors enhancing learning.
Factors enhancing learning
Learning environment
Testing
Figure 1.3 Learning and testing relationship
There must be a great number of factors enhancing learning, one of which is learning
environment. There must be a lot of things which constitute learning environment.
And testing is just a part of it. By all this I mean that if language testing is highly
problematic, it is in itself a bad learning environment which in turn may hurt
learning.
Further, as we are all aware of, language testing as an important phenomenon
in language learning has a number of stakes-holders or parties who are concerned in
it, ranging from local community such as the student test takers, public authority
such as the government to international community, such as authors and test
specialists. This is illustrated in Figure 1.4.
8
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
Government
Test
specialists
Schools
LANGUAGE
TESTING
Society
Educators,
Teachers
Parents
Students
Figure 1.4 Language testing and stakeholders
Since students and teachers, the most concerned stakeholders, deserve
benefits from testing, instead of colonization and oppression, time has come to
include their participation in the reform. It is necessary to do a research project
which enables to amplify and hear their voices as the starting point to shift the focus
from dealing with the nature of language, the nature of language use, the nature of
measurement, to understanding of the nature of learning.
In order to investigate the problems thoroughly, it is necessary to employ a
research paradigm which puts its interests in the participants’ narratives and enables
to go deeper by digging out their experience to the research topic; a research with
critical theory as the theoretical perspectives, which are concerned with empowering
human beings to transcend the constraints placed on them by race, class and gender
(Fay, 1987 as quoted in Creswell, 2003:10). And qualitative research with advocacy
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
approach is proved to be the right answer. Unlike quantitative research, which has
interests in making generalization, qualitative research, as literature indicates, does
not mean to make any generalization, for human beings cannot be generalized. A
number of methods are available to choose, ranging from, observation, open-ended
interviews, semi-structured interviews, structured interviews and documentary
analysis. More detailed description on the progressive qualitative research methods
will be available in the next chapter.
C. PROBLEM LIMITATION
To anticipate any probable problem which most likely arose from the
research due to various constraints such as the limited time, cost and my own
personal capacity, it was necessary to set up boundaries by using delimitation and
limitation as parameters for my study (cf. Creswell, 2003:147). I had to narrow down
the coverage (scope) and methodology of this study and identify any potential
weakness of the study.
From the numerous problems in language testing and learning already
identified, I wanted to focus my study more specifically on English testing and
learning. I limited the scope of English test here only to that conducted at the
vocational school consisting of classroom test and high-stakes tests with limited
aspects of goal, process and material. The stakeholders were limited to students and
teachers, with the reason that they represented local community to include in the
dialog and while they were the key factor in the reform, so far their involvement was
very low and inadequate as already indicated. To put it in another way, the
significant role of their lived experience and attitudes toward English testing had
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
regretfully ignored in both research and practice. The scope for teachers was limited
to those who teach English at the vocational school, while the scope for students
here is only to those who major in business studies at the vocational school. As to
methodology, I limited to those which are applied in the qualitative research. I chose
phenomenology as the strategy of enquiry with narrative designs. Data collecting
instruments such as interview, observation and document review would be
applicable.
The implication of this is that the research results would be interpretable
within the limit and the findings could be subject to other interpretations.
D. PROBLEM STATEMENT
Testing has been one of the most problematic areas in English education.
English testing has been used and designed for its own right and its own sake.
Learner’s learning factors have received less attention than they should. It is obvious
up to this point that in order to achieve the purpose of education as suggested by the
process model of curriculum in the progressivism philosophy, learning enhancement,
as already discussed, is urgent to implement. To ensure the learning enhancement,
one of the best ways is doing research in the problematic areas.
In order to be aware of and understand the students’ lived experience in
English testing at the vocational school, this research, therefore, aimed to answer
theoretically and empirically the following question, what is the vocational school
students’ lived experience in English testing and learning?. Therefore, throughout
the research we discussed and tried to seek the answer to how vocational school
students of Business Studies experience their English language testing and learning.
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
E. RESEARCH GOALS
This research aimed to describe, and interpret, how vocational school
students experience English testing and what in their perspective constitutes a good
English test and to reveal the answer to the research question theoretically and
experientially. As qualitative research, it was not meant to explain, predict, and
control the participant properties and behavior (cf. Creswell, 2003:31). Instead, it
was to reveal, describe and understand better how the vocational school students
experience English testing and to help them understand their own perspective, in
order to advocate becoming more empowered, self-fulfilling, and autonomous,
which means they can do what they are supposed to do and they can do it on their
own when they learn (cf. Finney, 2002:73). The goal of this research was therefore
to describe and interpret the students’ lived experience in English testing around the
time they learn English.
F. RESEARCH BENEFITS
This research, in general, will surely be beneficial to the evolving body of
knowledge in English education, science and technology. First, it will contribute
further insights for responding to problems regarding the use and development of
English test not for its own sake but in relation to what ESL is concerned with:
learning; and second, it will help educational policy makers and test specialists in
relation to course design and more specifically to the use and design of English tests.
The research, which comprises t
ABSTRACT
Neny Mariani, 2009. English Testing and Learning at the Vocational School:
Student’s Lived Experience. Yogyakarta: Sanata Dharma University, the
Graduate Program in English Studies
In the world of English education, in both theory and practice, testing which is
known as a means to measure learners’ abilities, continually poses big problems.
In the national level, testing has also become the main issue of a continual debate
and never-ending polemic. Despite all the shortcomings it may bring with it,
testing is widely used since it serves enormous practical functions. In this
situation, there is a need to promote learner-centered learning in that learning and
learner’s factors should be taken into consideration in the use and design of
English tests. This enhancement will accelerate learning autonomy; improve
quality and sustainability of learning process and achievement.
This research aimed at describing, interpreting and explaining how the vocational
school students experience English testing, revealing what testing means to them
and what in their perspective constitutes an ideal English test. As progressive
qualitative research, it was not meant to explain, predict, and control the student
and teacher properties and behavior. Instead, it was to reveal, describe and
understand better how the vocational school students experience English testing.
In order not to marginalize the participants as a result of the inquiry, this research
adopted the advocacy/participatory approach by starting with the suppression and
oppression issues and assumed that as the researcher I would proceed
collaboratively. Data were taken from an Indonesian vocational school with three
students (primary) and two English teachers (secondary) as data acquisition
sources. Data collection instruments included observation, interview and
document reviews. Verbal data, the participants’ narratives, were analyzed with
content analysis comprising descriptive and interpretative. To improve
trustworthiness of the research, data and methodological triangulation and thick
description were implemented over the course of the study. As a researcher, I
took an active participation and intervened by designing an on-going evaluation
in order to advocate becoming more empowered, autonomous, and self-fulfilling.
The research reveals that despite the similar concerns among the student
participants in testing, learning and achievement, there were unique emotional
experience and learning experience. It also reveals emergent themes around the
time of English testing which, in addition to the prefigured themes test backwash,
qualities, test content, learning and autonomy, includes status of test, fairness,
time allocation, achievement, competitiveness, preparedness. In addition, my
interpretation of what testing meant to the students as well as teachers shows that
there was a shift of concerns among teacher participants in different types within
xiv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
different contexts of testing. In the context of classroom testing, the teacher
participants had different concerns from the student participants, validity and
reliability, while the students were concerned about learning and achievement. In
the context of high-stakes testing, however, teachers and students shared similar
concerns, learning and achievement. That classroom testing potentially created a
point of clash and hurt student learning as its implication was due to the different
concerns in testing between the teachers and students. That high-stakes testing
created a gentleman’s agreement was due to the similar concerns. These research
results will lead to better understanding for the participants, students and
teachers, researcher as well as the audience. The participant improved awareness
of testing and learning and the researcher and audience improved emphatic
understanding of the participant experience, would consequently help facilitate
learner’s learning need and in turn lead to healthy learning process and better
results such as in particular to design a more progressive test and make a wiser
decision on why, when, how to use an educative test as assessment.
xv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
INTISARI
Neny Mariani, 2009. English Testing and Learning at the Vocational School:
Student’s Lived Experience. Yogyakarta: Universitas Sanata Dharma,
Kajian Bahasa Inggris
Di dalam pengajaran bahasa Inggris, baik dalam teori maupun praktek, tes yang
dikenal sebagai alat ukur kemampuan siswa masih menghadapi berbagai
masalah. Di kancah nasional, tes juga menjadi perdebatan dan polemik yang tak
kunjung selesai. Dengan segala kekurangannya, tes masih digunakan secara luas
karena masih banyak fungsinya. Dalam situasi seperti ini, diperlukan upaya
peningkatan pembelajaran yang berpusat pada pemelajar dalam hal faktor-faktor
kegiatan belajar dan siswa perlu diperhitungkan dalam penyusunan dan
pemakaian tes. Penekanan ini akan membawa dampak percepatan pembentukan
kemandirian belajar, meningkatkan kualitas dan keberlangsungan proses serta
keberhasilan proses dan prestasi belajar.
Penelitian ini mencoba menggambarkan, memahami, dan menjelaskan
bagaimana pengalaman siswa SMK dalam tes Bahasa Inggris dan mengungkap
apa makna tes bagi mereka serta apa pandangannya yang merupakan tes ideal itu.
Karena bersifat kualitatif, penelitian ini tidak dimaksudkan untuk menerangkan,
memprakirakan dan mengendalikan unsur-unsur yang ada pada siswa dan guru,
termasuk perilakunya. Namun, penelitian ini akan membuka tabir,
menggambarkan dan memahami secara lebih baik bagaimana pengalaman siswa
SMK terhadap tes Bahasa Inggris.
Untuk tidak mengesampingkan peserta sebagai akibat dari penelitian ini,
pendekatan advocacy/participatory digunakan dengan dimulai adanya isu
penekanan dan penindasan, dan anggapan bahwa peneliti akan melakukan
tugasnya dengan kompromi. Data diperoleh dari sebuah SMK di Indonesia
dengan sumber data dari tiga siswa (data primer) dan dua guru Bahasa Inggris
(data sekunder). Alat pengumpulan data berupa observasi, wawancara, dan telaah
dokumen. Data verbal yang berupa ucapan peserta dianalisis dengan analisis isi
yang meliputi analisis deskriptif and interpretatif. Untuk meningkatkan
keterpercayaan hasil penelitian, triangulasi data dan metodologi serta thick
description dilakukan selama proses dan pelaporan hasil penelitian. Sebagai
peneliti, saya terlibat aktif mengambil bagian dan melakukan intervensi dengan
cara melakukan refleksi kegiatan yang terus menerus dalam membantu
menyuarakan agar lebih berdaya, mandiri dan dapat memenuhi kebutuhan
belajarnya sendiri.
Penelitian menunjukkan bahwa meskipun di antara sesama siswa terdapat
persamaan dalam kepentingan yaitu belajar dan prestasi, mereka memiliki
perbedaan dalam pengalaman emosional dan pengalaman belajar. Penelitian ini
juga menunjukkan munculnya tema-tema baru seputar pelaksanaan tes yaitu
xvi
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
selain dampak, kualitas, isi tes, belajar dan kemandirian, juga mencakup status
tes, keadilan, alokasi waktu, prestasi, kebersaingan, dan kesiapan. Sebagai
tambahan, interpretasi saya mengenai apa makna tes bagi siswa dan guru SMK
menunjukkan adanya pergeseran kepentingan guru bila terdapat perbedaan jenis
dan konteks pengujian. Dalam konteks tes berbasis kelas, guru mempunyai
perbedaan kepentingan dengan siswa, yaitu validitas dan reliabilitas; sementara
itu siswa mempunyai kepentingan belajar dan prestasi. Namun dalam konteks
tes skala besar, guru dan siswa mempunyai kepentingan yang serupa, yaitu
belajar dan prestasi. Bahwa tes berbasis kelas berpotensi menimbulkan konflik
dan menghambat proses belajar siswa sebagai implikasinya itu terkait adanya
perbedaan kepentingan dalam tes antara guru dan siswa tersebut. Tes berskala
besar menimbulkan adanya saling pengertian itu karena kesamaan kepentingan.
Hasil penelitian ini akan memberikan pemahaman yang lebih baik bagi peserta,
siswa dan guru, peneliti dan khalayak. Meningkatnya kesadaran diri peserta
terhadap pengujian dan pemelajaran, dan rasa empati dari peneliti dan khalayak
akan membawa ke arah proses belajar yang sehat dan hasil yang lebih baik, dan
membantu mengakomodasi kepentingan dan kebutuhan belajar siswa, misalnya
dengan merancang tes yang bersifat progresif serta melakukan kebijakan tentang
mengapa, kapan, bagaimana menggunakan tes sebagai alat penilaian yang
mendidik.
xvii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
ENGLISH TESTING AND LEARNING
AT THE VOCATIONAL SCHOOL:
STUDENT’S LIVED EXPERIENCE
A THESIS
Presented in Partial Fulfillment of the Requirements
for the Master’s Degree
in English Language Studies
by
Neny Mariani
Student Number: 056332027
THE GRADUATE PROGRAM IN ENGLISH STUDIES
SANATA DHARMA UNIVERSITY YOGYAKARTA
2009
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
i
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
ii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
STATEMENT OF ORIGINALITY
This is to certify the originality of the written study. No single part of this thesis
but materials cited as references, either explicitly or implicitly, shows similarities or
resemblance with any other work, and therefore to the best of my knowledge, no
copyright violation is indicated. Hereby I declare that this thesis represents my
own work. I assume full responsibility for the content of the work and bear all
consequences it may result in.
Yogyakarta,
21st December 2009
Neny Mariani
iii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
LEMBAR PERNYATAAN
PERSETUJUAN PUBLIKASI KARYA ILMIAH
UNTUK KEPENTINGAN AKADEMIS
Yang bertanda tangan di bawah ini, saya mahasiswa Universitas Sanata Dharma
Yogyakarta dengan
Nama
: Neny Mariani
Nomor mahasiswa: 056332027
menyatakan bahwa demi pengembangan ilmu pengetahuan memberikan kepada
Perpustakaan Universitas Sanata Dharma Yogyakarta karya ilmiah yang berjudul:
“English Testing and Learning at the Vocational School: Student’s Lived Experience”
yang saya tulis beserta perangkat terkait yang tersedia.
Dengan demikian kepada Perpustakaan Universitas Sanata Dharma Yogyakarta
saya berikan hak dan wewenang untuk menyimpan, mengalihkan dalam bentuk media
lain, mengolahnya dalam bentuk pangkalan data, mendistribusikan secara terbatas, dan
mempublikasikannya di internet atau media lain untuk kepentingan akademis tanpa perlu
meminta ijin dari saya maupun memberikan royalti kepada saya selama tetap
mencantumkan nama saya sebagai penulis.
Demikian pernyataan ini saya buat dengan sesungguhnya.
Dibuat di
: Yogyakarta
Pada tanggal: 21 Desember 2009
Yang menyatakan,
Neny Mariani
iv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
ACKNOWLEDGMENTS
First of all, I wish to offer my biggest adoration to the Almighty, who has
granted me an opportunity to pursue further education and kept sending so many
wonderful people I learn from and work with. I am so grateful for all the blessings,
inspiration and never-ending guide from nothing to something, prior to, during, and
upon completion of this master’s thesis. I guess there will never be bigger celebration to
mark the accomplishment of this project than expressing the words of glory, "Praise be
to Allah, the Cherisher, Sustainer of the worlds!". Also, I do not want to miss this
opportunity to express a word of honor to my parents, from whom I received my first
education in my earliest years, for raising me up and introducing the Most Gracious and
Merciful.
Although the content of this thesis is essentially and eventually my own
responsibility, this work by all means results from a great number of contributors.
Therefore I wish to take this opportunity to acknowledge and appreciate all kinds of
contribution which I have received directly and indirectly from those whose names I will
not be able to print out here one by one.
First of all, I would extend my particular gratitude to my adviser, Pak Bis (Dr J
Bismoko), for his great dedication and allowing me to get an easy access for correction
and consultation. It would almost be impossible to do this research project without the
existence of such a prominent figure behind the preparation, implementation and
presentation of the work.
I have been largely indebted to all the lecturers and everyone in the graduate
v
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
program of this university. I would like to thank the Graduate Program Director, Prof.
Dr. Augustinus Supratiknya for legalizing this report. I thank the Board of Examiners,
Pak Muk (FX. Mukarto, Ph.D), Pak Dwi (Dr. B.B. Dwijatmoko, M.A), Pak Alip (Dr. Fr.
B. Alip, M.Pd, M.A.) for helping me finalize this work with the thorough examination. I
also thank the rest of the lecturers, Pak Barli (Drs. Barli Bram M.S.), Bu Novi (Dr.
Novita Dewi, M.S., M.A. (Hons)), and Pak Pomo (Prof. Dr. Soepomo Poedjosoedarmo)
for their "shower of sparks" and "drops of dews".
For the financial and non financial support as well as attention, my gratitude goes
to LPMP Yogyakarta, including Bu Anna (Dra. Anna Maria Hendraswari, M.A.), Pemda
Kabupaten Kulon Progo (including Pak Ridwan) and Dinas Pendidikan Kabupaten
Kulon Progo (including Pak Yatiman ).
Next, I would express my sincere gratitude to my research participants, my
fellow English teachers and students, for every word they have contributed, their
honesty, and the time and effort they have spent to reply to my enquiries. For me, they
have been sunshine in the rainy day and water in the desert. I would like to express my
thankfulness to everybody in the school where I spend wonderful years, SMK Negeri 1
Pengasih, which includes the school principal Drs. H. Rumawal, my colleagues and
students, for the encouragement, challenge, spiritual, technical and financial support.
I thank all authors and researchers whose work is very valuable and contributive
to my project. I have made every effort to trace all the resources the best I could, and I
hope I do not miss a thing.
To all my fellow students and my acquaintances, I would say a big thank-you for
the share of knowledge and feelings. All the years when we were involved in such tough
vi
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
and fierce discussions, arguing for and against, sharing laughter and tears have been so
precious to me. I will always miss the time when the feeling of friendship was that
strong.
Upon completion of the work, I would like to dedicate this thesis to the
Motherland and to the world. I realize this work will be more meaningful it gains
attention from wider audience. I look forward to and welcome any response coming
from all directions.
Yogyakarta, 21st December 2009
Neny Mariani
vii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
TABLE OF CONTENTS
TITLE PAGE
APPROVAL PAGE
DEFENSE APPROVAL PAGE
STATEMENT OF ORIGINALITY
STATEMENT OF LICENCE AGREEMENT
ACKNOWLEDGEMENTS
TABLE OF CONTENTS
LIST OF FIGURES
LIST OF TABLES
LIST OF ABBREVIATIONS
ABSTRACT
INTISARI
CHAPTER I
A.
B.
C.
D.
E.
F.
1
INTRODUCTION
BACKGROUND
PROBLEM IDENTIFICATION
PROBLEM LIMITATION
PROBLEM STATEMENT
RESEARCH GOALS
RESEARCH BENEFIT
CHAPTER II
i
ii
iii
iv
v
viii
xi
xii
xiii
xiv
xvi
1
5
10
11
12
12
LITERATURE REVIEW
14
A. THEORETICAL REVIEW
1. Assessment
2. Language testing, English testing and backwash effects
3. Learning and learning autonomy
4. Language learning and language learning autonomy
5. Vocational school
6. Experience and attitude
7. Qualitative research and methodologies
B. THEORETICAL FRAMEWORK
14
14
18
29
34
37
39
40
46
CHAPTER III
55
A.
B.
C.
D.
E.
METHODOLOGY
RESEARCH DESIGN
DATA SOURCES
TYPES OF DATA
DATA COLLECTION TECHNIQUES
DATA PROCESSING PROCEDURE
1. DATA COLLECTING
2. DATA ANALYZING
viii
55
60
62
63
66
66
67
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
F. TRUSTWORTHINESS OF RESEARCH RESULTS
CHAPTER IV
.
ANALYSIS RESULTS
A. DATA AND SOURCE SELECTION
B. INSTRUMENT VALIDATION AND ACCOUNTABILITY
C. DATA GATHERING, RECORDING AND PRESENTATION
D. DATA ANALYSIS ACTUALIZATION
E. DISCUSSION: DESCRIPTION AND INTERPRETATION
1. Student participant portraits in English testing and learning
2. Thematic description and illustrative quotations
3. Participants’ meaning of testing
129
CHAPTER V
CONCLUSIONS, IMPLICATIONS,
AND RECOMMENDATIONS
A. CONCLUSIONS
B. IMPLICATIONS
C. RECOMMENDATIONS
68
73
73
78
80
85
106
106
115
144
144
148
149
BIBLIOGRAPHY
151
APPENDICES
158
Appendix 1 Interview probes
1.1 Questions to students
1.2 Questions to teachers
Appendix 2 Coding blue print
2.1 Participant: students
2.2 Participant: teachers
2.3 Observation on test implementation
2.4 Inductive codes
Appendix 3 Observation data of events and behavior from official meetings
and environs
3.1 Official meeting 1 ( external)
3.2 Official meeting 2 ( external)
3.3 Official meeting 3 ( external)
3.4 Official meeting 4 (internal)
Appendix 4 Observation data of events and behavior from test sites
4.1 Classroom test 1
4.2 Classroom test 2
4.3 Classroom test 3
4.4 Classroom test 4
4.5 Classroom test 5
4.6 High-stakes test 1
4.7 High-stakes test 2
ix
158
158
158
159
159
160
161
161
162
162
163
165
165
166
166
166
167
167
167
168
169
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
4.8 High-stakes test 3
Appendix 5 Observation data of physical and non physical environs
5.1 Situation prior to classroom test 1
5.2 Situation prior to classroom test 2
5.3 Situation prior to high-stakes test 1
5.4 Situation prior to high-stakes test 2
5.5 Situation after high-stakes test 1
5.6 Situation after high-stakes test 2
5.7 Situation after high-stakes test 3
Appendix 6 Researcher’s diary
Appendix 7 Interview data
7.1 Student Atik 1
7.2 Student Atik 2
7.3 Student Siti 1
7.4 Student Siti 2
7.5 Student Endang
7.6 Teacher Iin 1
7.7 Teacher Iin 2
7.8 Teacher Nanik 1
7.9 Teacher Nanik 2
Appendix 8. News-stories and information
8.1 Radio news
8.2 Television news
8.3 Printed news
8.3.1 Printed news 1
8.3.2 Printed news 2
Appendix 9 Documents
9.1 Official document 1
9.2 Official document 2
9.3 Student Endang’s answer sheet
9.4 Student Atik’s work
9.5 Student Atik’s school report
9.6 Classroom test sheet
9.7 High-stakes test booklet
9.8 Test-result report
9.8.1 Score report from high-stakes test 1
9.8.2 Score report from high-stakes test 2
x
170
171
171
172
172
173
174
174
174
175
183
183
191
201
207
214
221
226
238
245
249
249
250
257
257
258
259
259
260
261
262
263
266
267
269
269
270
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
LIST OF FIGURES
1. Figure 1.1 Tripartite partnership model
2. Figure 1.2 First, essential stage in learning enhancement partnership
3. Figure 1.3 Learning and testing relationship
4. Figure 1.4 Testing and stakeholders
5. Figure 2.1 Communication in testing environment stage 1, 2, 3
6. Figure 2.2 Bachman’s categorization of the use of tests
7. Figure 2.3 Brown’s language test type categorization
8. Figure 2.4 McNamara’s language test type categorization
9. Figure 2.5 Test type categorization
10. Figure 2.6 Testing in the language program toward independent learning
11. Figure 2.7 Positioning English testing at the Vocational School
12. Figure 2.8 English testing aspects categorization
13. Figure 3.1 From research questions to research results
14. Figure 3.2 Types of data
15. Figure 3.3 Data processing procedure
16. Figure 4.1 Student’s lived experience in English testing and learning
at the vocational school seen from various angles
17. Figure 4.2 Data processing stages
18. Figure 4.3 High-stakes test and thematic description, gleaned from
secondary data sources
19. Figure 4.4 Different interests within testing
xi
6
7
8
9
21
25
27
27
29
47
50
52
59
62
66
79
87
118
140
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
LIST OF TABLES
Table 2.1 Theoretical concept map
Table 3.1 Research specification
Table 4.1 Student participant data
Table 4.2 Teacher participant data
Table 4.3 Observation blueprint
Table 4.4 Sample data observation 1
Table 4.5 Sample data observation 2
Table 4.6 Probing interview questions
Table 4.7 A priori coding inventory for student participants
Table 4.8 Sample corpus of data
Table 4.9 Sample tabulating table
Table 4.10 Semi-ordered list of responses to the open-ended question
“Bagaimana perasaanmu sebelum, pada saat, dan setelah
mengikuti ulangan harian?”
Table 4.11 Category of responses to the open-ended question
“Bagaimana perasaanmu sebelum, pada saat, dan
setelah mengikuti ulangan harian?”
Table 4.12 Emerging themes and categories from student participants’
narratives and behavior
Table 4.13 Components, categories and indicators constituting
environment around English testing implementation
Table 4.14 Emergent categories from primary and secondary data
xii
54
58
76
76
81
82
83
84
85
88
96
101
103
104
116
117
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
LIST OF ABBREVIATIONS
CARLA: Center for Advanced Research on Language Acquisition
Depdiknas: Departemen Pendidikan Nasional
EFL: English as a Foreign Language
ESL: English as a Second Language
ESP: English for Specific Purpose
FHI: Family Health International
LOTE: Language Other Than English
MTs: Madrasah Tsanawiyah
RRI: Radio Republik Indonesia
SLA: Second Language Acquisition
SMK: Sekolah Menengah Kejuruan
SMP: Sekolah Menengah Pertama
TOEIC: Test of English for International Communication
TVRI: Televisi Republik Indonesia
UWE: University of the West England
xiii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
ABSTRACT
Neny Mariani, 2009. English Testing and Learning at the Vocational School:
Student’s Lived Experience. Yogyakarta: Sanata Dharma University, the
Graduate Program in English Studies
In the world of English education, in both theory and practice, testing which is
known as a means to measure learners’ abilities, continually poses big problems.
In the national level, testing has also become the main issue of a continual debate
and never-ending polemic. Despite all the shortcomings it may bring with it,
testing is widely used since it serves enormous practical functions. In this
situation, there is a need to promote learner-centered learning in that learning and
learner’s factors should be taken into consideration in the use and design of
English tests. This enhancement will accelerate learning autonomy; improve
quality and sustainability of learning process and achievement.
This research aimed at describing, interpreting and explaining how the vocational
school students experience English testing, revealing what testing means to them
and what in their perspective constitutes an ideal English test. As progressive
qualitative research, it was not meant to explain, predict, and control the student
and teacher properties and behavior. Instead, it was to reveal, describe and
understand better how the vocational school students experience English testing.
In order not to marginalize the participants as a result of the inquiry, this research
adopted the advocacy/participatory approach by starting with the suppression and
oppression issues and assumed that as the researcher I would proceed
collaboratively. Data were taken from an Indonesian vocational school with three
students (primary) and two English teachers (secondary) as data acquisition
sources. Data collection instruments included observation, interview and
document reviews. Verbal data, the participants’ narratives, were analyzed with
content analysis comprising descriptive and interpretative. To improve
trustworthiness of the research, data and methodological triangulation and thick
description were implemented over the course of the study. As a researcher, I
took an active participation and intervened by designing an on-going evaluation
in order to advocate becoming more empowered, autonomous, and self-fulfilling.
The research reveals that despite the similar concerns among the student
participants in testing, learning and achievement, there were unique emotional
experience and learning experience. It also reveals emergent themes around the
time of English testing which, in addition to the prefigured themes test backwash,
qualities, test content, learning and autonomy, includes status of test, fairness,
time allocation, achievement, competitiveness, preparedness. In addition, my
interpretation of what testing meant to the students as well as teachers shows that
there was a shift of concerns among teacher participants in different types within
xiv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
different contexts of testing. In the context of classroom testing, the teacher
participants had different concerns from the student participants, validity and
reliability, while the students were concerned about learning and achievement. In
the context of high-stakes testing, however, teachers and students shared similar
concerns, learning and achievement. That classroom testing potentially created a
point of clash and hurt student learning as its implication was due to the different
concerns in testing between the teachers and students. That high-stakes testing
created a gentleman’s agreement was due to the similar concerns. These research
results will lead to better understanding for the participants, students and
teachers, researcher as well as the audience. The participant improved awareness
of testing and learning and the researcher and audience improved emphatic
understanding of the participant experience, would consequently help facilitate
learner’s learning need and in turn lead to healthy learning process and better
results such as in particular to design a more progressive test and make a wiser
decision on why, when, how to use an educative test as assessment.
xv
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
INTISARI
Neny Mariani, 2009. English Testing and Learning at the Vocational School:
Student’s Lived Experience. Yogyakarta: Universitas Sanata Dharma,
Kajian Bahasa Inggris
Di dalam pengajaran bahasa Inggris, baik dalam teori maupun praktek, tes yang
dikenal sebagai alat ukur kemampuan siswa masih menghadapi berbagai
masalah. Di kancah nasional, tes juga menjadi perdebatan dan polemik yang tak
kunjung selesai. Dengan segala kekurangannya, tes masih digunakan secara luas
karena masih banyak fungsinya. Dalam situasi seperti ini, diperlukan upaya
peningkatan pembelajaran yang berpusat pada pemelajar dalam hal faktor-faktor
kegiatan belajar dan siswa perlu diperhitungkan dalam penyusunan dan
pemakaian tes. Penekanan ini akan membawa dampak percepatan pembentukan
kemandirian belajar, meningkatkan kualitas dan keberlangsungan proses serta
keberhasilan proses dan prestasi belajar.
Penelitian ini mencoba menggambarkan, memahami, dan menjelaskan
bagaimana pengalaman siswa SMK dalam tes Bahasa Inggris dan mengungkap
apa makna tes bagi mereka serta apa pandangannya yang merupakan tes ideal itu.
Karena bersifat kualitatif, penelitian ini tidak dimaksudkan untuk menerangkan,
memprakirakan dan mengendalikan unsur-unsur yang ada pada siswa dan guru,
termasuk perilakunya. Namun, penelitian ini akan membuka tabir,
menggambarkan dan memahami secara lebih baik bagaimana pengalaman siswa
SMK terhadap tes Bahasa Inggris.
Untuk tidak mengesampingkan peserta sebagai akibat dari penelitian ini,
pendekatan advocacy/participatory digunakan dengan dimulai adanya isu
penekanan dan penindasan, dan anggapan bahwa peneliti akan melakukan
tugasnya dengan kompromi. Data diperoleh dari sebuah SMK di Indonesia
dengan sumber data dari tiga siswa (data primer) dan dua guru Bahasa Inggris
(data sekunder). Alat pengumpulan data berupa observasi, wawancara, dan telaah
dokumen. Data verbal yang berupa ucapan peserta dianalisis dengan analisis isi
yang meliputi analisis deskriptif and interpretatif. Untuk meningkatkan
keterpercayaan hasil penelitian, triangulasi data dan metodologi serta thick
description dilakukan selama proses dan pelaporan hasil penelitian. Sebagai
peneliti, saya terlibat aktif mengambil bagian dan melakukan intervensi dengan
cara melakukan refleksi kegiatan yang terus menerus dalam membantu
menyuarakan agar lebih berdaya, mandiri dan dapat memenuhi kebutuhan
belajarnya sendiri.
Penelitian menunjukkan bahwa meskipun di antara sesama siswa terdapat
persamaan dalam kepentingan yaitu belajar dan prestasi, mereka memiliki
perbedaan dalam pengalaman emosional dan pengalaman belajar. Penelitian ini
juga menunjukkan munculnya tema-tema baru seputar pelaksanaan tes yaitu
xvi
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
selain dampak, kualitas, isi tes, belajar dan kemandirian, juga mencakup status
tes, keadilan, alokasi waktu, prestasi, kebersaingan, dan kesiapan. Sebagai
tambahan, interpretasi saya mengenai apa makna tes bagi siswa dan guru SMK
menunjukkan adanya pergeseran kepentingan guru bila terdapat perbedaan jenis
dan konteks pengujian. Dalam konteks tes berbasis kelas, guru mempunyai
perbedaan kepentingan dengan siswa, yaitu validitas dan reliabilitas; sementara
itu siswa mempunyai kepentingan belajar dan prestasi. Namun dalam konteks
tes skala besar, guru dan siswa mempunyai kepentingan yang serupa, yaitu
belajar dan prestasi. Bahwa tes berbasis kelas berpotensi menimbulkan konflik
dan menghambat proses belajar siswa sebagai implikasinya itu terkait adanya
perbedaan kepentingan dalam tes antara guru dan siswa tersebut. Tes berskala
besar menimbulkan adanya saling pengertian itu karena kesamaan kepentingan.
Hasil penelitian ini akan memberikan pemahaman yang lebih baik bagi peserta,
siswa dan guru, peneliti dan khalayak. Meningkatnya kesadaran diri peserta
terhadap pengujian dan pemelajaran, dan rasa empati dari peneliti dan khalayak
akan membawa ke arah proses belajar yang sehat dan hasil yang lebih baik, dan
membantu mengakomodasi kepentingan dan kebutuhan belajar siswa, misalnya
dengan merancang tes yang bersifat progresif serta melakukan kebijakan tentang
mengapa, kapan, bagaimana menggunakan tes sebagai alat penilaian yang
mendidik.
xvii
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
CHAPTER I
INTRODUCTION
To identify and expose the main research problems elaborately, in this
introductory chapter I will examine concepts and issues in English education,
specifically English testing in the global as well as local context. I will begin with the
background of the study and consecutively discuss the problem identification, problem
limitation, problem statement, research goals, and research benefits.
A. BACKGROUND
Over the last few decades, as writers say, language test theory has put a great
deal of emphasis on three major areas. First is the nature of language; that is when we
talk about skills and language components. Next is the nature of language use; that is
when we talk about authenticity. Third is the nature of measurement; that is when we
talk about validity and reliability. This can be seen from the work of, take for example,
Bachman (1995) and McNamara (1996). It is not surprising then that in practice test
developers and specialists are reportedly deeply concerned with anything related to
defining language competence, understanding the factors which influence performance
on language tests and finding valid and reliable ways of measuring language competence
and development, and recently greatly worried about whether test materials and tasks
represent actual use of language for communication. In conclusion, we are excessively
concerned with the overt function of the test, as a means of measurement and
assessment, ignoring the covert but influential function, that is, as a learning experience.
1
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
It is clear for us that most test theories as proposed by Bachman (1995),
McNamara (1996) and Brown (2005) imply that they place test-takers as the objects
of the test. Test-takers are considered as “something we can do measurement upon”.
It is not clear whether it will give any benefits or harms to them. The definitions
imply that test is designed for the sake of other parties, while the primary
stakeholders, which are in fact the test-takers themselves, are totally forgotten. This
is what I call oppression, suppression and colonization. Up to this point, it is
obvious that so far we have been busy dealing with constructing a test for its own
sake and for its own right, rather than catering for student learning needs or
facilitating learning in the stage of evaluation within the teaching and learning
process. Whether the test construction has harmful or positive effects on learning
remains out of discussion and little or even almost nothing about learning has been
taken into consideration in the design and use of language tests in most EFL/ESL
courses. The lack of discussion and the absence of such consideration in the test
design and use is greatly regrettable. In relation to this, Hughes’ (2003:1) assertion,
showing the inadequacy in the test design and use, is worth quoting, “[…] a great
deal of language testing is of very poor quality. Too often language tests have
harmful effects on teaching and learning […]”.
In this problematic situation, we then need a review on the most current
educational and learning issues. Life-long education has been widely noted since
early times and Delors et al. (1998:20) reaffirm it by iterating that the concept of
education pursued throughout life recently emerges as one of the keys to the twenty
first century. Education, an ongoing process of improving knowledge, skills and
attitude or behavior, with the four pillars on which education throughout life is
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
based, learning to know, learning to do, learning to be and learning to live together,
is said to have a fundamental role to play in personal and social development in that
it is an exceptional means of bringing about personal development and building
relationship among individuals, groups and nations.
Next, let us refer to the postmodernism theory, with its progressivism
philosophy, which says education should be directed to make learners empowered,
autonomous, and eventually become self-fulfilling individually and socially in their
own world. Finney (2002:73) puts it this way,
The purpose of education from the point of view of the process model is to
enable the individual to progress toward self-fulfillment. It is concerned with
the development of understanding, not just the passive reception of
‘knowledge’ or the acquisition of specific skills. The goals of education are
not defined in terms of particular ends or products, but in terms of the
processes and procedures by which the individual develops understanding
and awareness and creates possibilities for future learning. (Italicized words
are my emphasis)
Here we should note that the key word to education is “self-fulfillment”, and
consequently we can say we need learner-centeredness, an approach to education
that aims at developing in each student a sense of responsibility for his or her own
learning (The University’s Strategic Plan, 1995 as quoted in Hall, 2000:2). In other
words it means the development of learner autonomy should be the heart of any
educational program, which certainly includes language teaching and learning.
To ensure the enhancement of learner-centered learning and the development
of learner autonomy as stated above, therefore, I felt it urgent to do research on a
particular topic so that learning, which is the most essential of all in education,
would be appropriately ensured, adequately addressed and consistently become the
main consideration in the whole process of designing and implementing educational
course all the way from assessing learner’s learning needs to the stage of program
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
evaluation, including testing.
Test, which is in a general sense the main instrument in learner assessment,
is indispensable part of the ESL/EFL/ESP teaching and learning process, and it has
been my interest to do a research on. The choice of the topic was due to the
following reasons. First, as you observe, test as the main instrument of assessment
remains controversial and problematic in both theory and practice in that students’
learning needs and viewpoints in testing has been paid little attention. In addition, we
can refer to that of two prominent advocates of ESP Hutchinson and Waters
(1994:146), who realize and notice there is a shortage of sound theoretical or
empirical basis for ESP testing and lack of discussion or guidance on ESP testing:
“Far more research is therefore needed before we will really know what the
requirements of a good ESP test are”. Second, my personal and professional
background as an EFL teacher had driven me conduct numerous kinds of tests for
vocational school students in the local, regional as well as national level. Thus, in
Trochim’s words (2006:1), I had been living and grown with the phenomenon, or in
other words, it is my “direct experience of the phenomenon”. Therefore in relation to
the learning enhancement and the learner autonomy development as I have already
described, then I concentrated on the topic, English testing.
In relation to the topic, I believed, the focus of further research on testing
should be on learning factors and an understanding of the impacts of language
testing to learning, instead of language or test property analysis. The statement of
Chalhoub-Deville (2003:380) is worth quoting to confirm the idea,
Given current indications in language testing, the nature of the components
underlying the L2 construct as well as the manner in which these components
interact are issues that researchers need to address in the twenty-first century.
4
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
Indonesian vocational schools, whose mission is to provide a middle-class
workforce and to improve their competitiveness, are greatly concerned with their
EFL or more specifically ESP teaching and learning. A policy shift from a supplydriven system to a demand-driven system guided by labor market signal, curriculum
reform, developing syllabus design, improvement on learning resources, teacher
training, the adoption of the international standardized test, periodical debate contest
are clear illustrations of the struggle. Nevertheless, the major common problem
remains. Having spent their years of schooling, a number of students are still
considered ‘unsuccessful’ English learners, not only emerging with a very low
proficiency but also indicating low motivation and having a high dependence on
teachers in their learning.
Thus, despite tremendous progress in curriculum reforms and policies, two
significant things, which are implications of the curriculum reforms itself, remain
untouched: understanding of student’s learning needs and experience in testing. In
such a situation, to do research on English testing and learning is highly relevant
since the results will surely be contributive to learning enhancement
and
sustainability and eventually help overcome the persistent problems in education.
B. PROBLEM IDENTIFICATION
The problematic situation, which takes place in both theory and practice,
implies a kind of oppression or colonization upon learners, and then it raises
questions as “If a test is a measurement, why measure?, “Is measuring the ultimate
goal in all education?”, ”Where is in language testing the room for learning
facilitation which builds learner’s autonomy?”, “Is a traditional paper and pencil
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
test totally inauthentic in language testing?”, “Does something inauthentic have no
meaning and bring bad impacts to learning?”, “If a test is an instrument for us to
use, why is it not designed as user-friendly?”. With all these problematic questions,
it is clear that an educational reform is urgent to do, and further research on English
testing is the right way to implement the reform.
To get out of the problematic situation in the world of education, it is a
good idea to refer to what Delors et al. (1998: 27) state,
The main parties contributing to the success of educational reforms are,
first of all, the local community, including parents, school heads and
teachers; secondly, the public authorities; and thirdly, the international
community. Many past failures have been due to insufficient involvement
of one or more of these partners. (bold-faced is my emphasis)
From here, it is obvious that the involvement of all parties in the reform is
obligatory; local community’s contribution takes precedence over the others, and
that the absence of any party in the involvement will lead to another failure. Figure
1.1 presents a clear illustration of this.
First Party
Local Community
Educational Reforms
Second Party
Public Authority
Third Party
International Community
Figure 1.1 Tripartite partnership model in education reform
(visualizing the idea of Delors et al., 1998 )
6
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
Furthermore, for the success of educational reforms, Delors et al. (1998:27)
also necessitate a dialog among the parties as they point out,
Local community participation in assessing needs by means a dialogue with
the public authorities and group concerned in society is a first, essential stage
in broadening access to education and improving its quality. (my emphasis)
This is in line with learner-centered teaching orientation which begins with the
premise that the learner is the central reference point in decision making (Tudor,
1997). Thus, if we want to enhance learner-centered learning and to know the
requirements of a good test, learning-needs assessment is significant to do, and
having a dialog with test stakeholders to ensure sufficient involvement of pu
blic authority and local community is highly relevant. This can be illustrated in
Figure 1.2.
Local Community:
Students, parents, teachers, school heads
Educational Reform:
Enhancing learner-centered learning
Public Authority:
Government, researchers
Figure 1.2 First, essential stage in learning enhancement
partnership (developed from the idea of Delors et al.,
1998)
One of the logical consequential applications of this theory is that it is urgent
to do research which cares about the learners’ concern by having a dialog with
students and teachers as the local community party. Again, Tudor (1997) confirms
this idea, stating that needs analysis and goal-setting can be more effective if
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
students inform them with their knowledge and experience; and learning is more
successful if methodology and study mode take into account student preferences and
characteristics. In this research, therefore I as the researcher representing the public
authority party will facilitate what they want to say in order to implement the reform
effectively.
To begin with, let us study how learning and testing is interrelated.
Here, Figure 1.3 shows that in the context of educational program, testing can be
seen as a learning environment, which constitutes one of factors enhancing learning.
Factors enhancing learning
Learning environment
Testing
Figure 1.3 Learning and testing relationship
There must be a great number of factors enhancing learning, one of which is learning
environment. There must be a lot of things which constitute learning environment.
And testing is just a part of it. By all this I mean that if language testing is highly
problematic, it is in itself a bad learning environment which in turn may hurt
learning.
Further, as we are all aware of, language testing as an important phenomenon
in language learning has a number of stakes-holders or parties who are concerned in
it, ranging from local community such as the student test takers, public authority
such as the government to international community, such as authors and test
specialists. This is illustrated in Figure 1.4.
8
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
Government
Test
specialists
Schools
LANGUAGE
TESTING
Society
Educators,
Teachers
Parents
Students
Figure 1.4 Language testing and stakeholders
Since students and teachers, the most concerned stakeholders, deserve
benefits from testing, instead of colonization and oppression, time has come to
include their participation in the reform. It is necessary to do a research project
which enables to amplify and hear their voices as the starting point to shift the focus
from dealing with the nature of language, the nature of language use, the nature of
measurement, to understanding of the nature of learning.
In order to investigate the problems thoroughly, it is necessary to employ a
research paradigm which puts its interests in the participants’ narratives and enables
to go deeper by digging out their experience to the research topic; a research with
critical theory as the theoretical perspectives, which are concerned with empowering
human beings to transcend the constraints placed on them by race, class and gender
(Fay, 1987 as quoted in Creswell, 2003:10). And qualitative research with advocacy
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
approach is proved to be the right answer. Unlike quantitative research, which has
interests in making generalization, qualitative research, as literature indicates, does
not mean to make any generalization, for human beings cannot be generalized. A
number of methods are available to choose, ranging from, observation, open-ended
interviews, semi-structured interviews, structured interviews and documentary
analysis. More detailed description on the progressive qualitative research methods
will be available in the next chapter.
C. PROBLEM LIMITATION
To anticipate any probable problem which most likely arose from the
research due to various constraints such as the limited time, cost and my own
personal capacity, it was necessary to set up boundaries by using delimitation and
limitation as parameters for my study (cf. Creswell, 2003:147). I had to narrow down
the coverage (scope) and methodology of this study and identify any potential
weakness of the study.
From the numerous problems in language testing and learning already
identified, I wanted to focus my study more specifically on English testing and
learning. I limited the scope of English test here only to that conducted at the
vocational school consisting of classroom test and high-stakes tests with limited
aspects of goal, process and material. The stakeholders were limited to students and
teachers, with the reason that they represented local community to include in the
dialog and while they were the key factor in the reform, so far their involvement was
very low and inadequate as already indicated. To put it in another way, the
significant role of their lived experience and attitudes toward English testing had
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
regretfully ignored in both research and practice. The scope for teachers was limited
to those who teach English at the vocational school, while the scope for students
here is only to those who major in business studies at the vocational school. As to
methodology, I limited to those which are applied in the qualitative research. I chose
phenomenology as the strategy of enquiry with narrative designs. Data collecting
instruments such as interview, observation and document review would be
applicable.
The implication of this is that the research results would be interpretable
within the limit and the findings could be subject to other interpretations.
D. PROBLEM STATEMENT
Testing has been one of the most problematic areas in English education.
English testing has been used and designed for its own right and its own sake.
Learner’s learning factors have received less attention than they should. It is obvious
up to this point that in order to achieve the purpose of education as suggested by the
process model of curriculum in the progressivism philosophy, learning enhancement,
as already discussed, is urgent to implement. To ensure the learning enhancement,
one of the best ways is doing research in the problematic areas.
In order to be aware of and understand the students’ lived experience in
English testing at the vocational school, this research, therefore, aimed to answer
theoretically and empirically the following question, what is the vocational school
students’ lived experience in English testing and learning?. Therefore, throughout
the research we discussed and tried to seek the answer to how vocational school
students of Business Studies experience their English language testing and learning.
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
E. RESEARCH GOALS
This research aimed to describe, and interpret, how vocational school
students experience English testing and what in their perspective constitutes a good
English test and to reveal the answer to the research question theoretically and
experientially. As qualitative research, it was not meant to explain, predict, and
control the participant properties and behavior (cf. Creswell, 2003:31). Instead, it
was to reveal, describe and understand better how the vocational school students
experience English testing and to help them understand their own perspective, in
order to advocate becoming more empowered, self-fulfilling, and autonomous,
which means they can do what they are supposed to do and they can do it on their
own when they learn (cf. Finney, 2002:73). The goal of this research was therefore
to describe and interpret the students’ lived experience in English testing around the
time they learn English.
F. RESEARCH BENEFITS
This research, in general, will surely be beneficial to the evolving body of
knowledge in English education, science and technology. First, it will contribute
further insights for responding to problems regarding the use and development of
English test not for its own sake but in relation to what ESL is concerned with:
learning; and second, it will help educational policy makers and test specialists in
relation to course design and more specifically to the use and design of English tests.
The research, which comprises t