ANALYTIC SCORING PROCEDURE IN SPEAKING ASSESSMENT AT HOSPITALITY PROGRAM OF STATE VOCATIONAL HIGH SCHOOL 1 BUDURAN SIDOARJO.

(1)

ANALYTIC SCORING PROCEDURE IN

SPEAKING ASSESSMENT AT HOSPITALITY

PROGRAM OF STATE VOCATIONAL HIGH

SCHOOL 1 BUDURAN SIDOARJO

THESIS

Submitted in Partial Fulfillment of the Requirement for the degree of Sarjana Pendidikan (S.Pd) in Teaching English

By:

RIZKY AMALIA INDAYANI NIM D35212054

ENGLISH TEACHER EDUCATION DEPARTMENT

FACULTY OF TARBIYAH AND TEACHER TRAINING

SUNAN AMPEL STATE ISLAMIC UNIVERSITY

SURABAYA

(2)

(3)

APPROVAL SHEET

This thesis by Rizky Amalia Indayani entitled “Analytic Scoring Procedure in Speaking Assessment at Hospitality Program of State Vocational High School 1 Buduran Sidoarjo” has been examined on February 3rd 2017 and approved by board of examiners:

(4)

(5)

LEMBAR PERNYATAAN PERSETUJUAN PUBLIKASI KARYA ILMIAH UNTUK KEPENTINGAN AKADEMIS

Sebagai sivitas akademika UIN Sunan Ampel Surabaya, yang bertanda tangan di bawah ini, saya:

Nama : RIZKY AMALIA INDAYANI

NIM : D35212054

Fakultas/Jurusan : FTK/PENDIDIKAN BAHASA INGGRIS E-mail address : rizkyamaliaindayani@yahoo.com

Demi pengembangan ilmu pengetahuan, menyetujui untuk memberikan kepada Perpustakaan UIN Sunan Ampel Surabaya, Hak Bebas Royalti Non-Eksklusif atas karya ilmiah :

Skripsi Tesis Disertasi Lain-lain (………) yang berjudul :

ANALYTIC SCORING PROCEDURE IN SPEAKING ASSESSMENT OF

HOSPITALITY PROGRAM AT STATE VOCATIONAL HIGH SCHOOL 1

BUDURAN SIDOARJO.

beserta perangkat yang diperlukan (bila ada). Dengan Hak Bebas Royalti Non-Ekslusif ini Perpustakaan UIN Sunan Ampel Surabaya berhak menyimpan, mengalih-media/format-kan, mengelolanya dalam bentuk pangkalan data (database), mendistribusikannya, dan menampilkan/mempublikasikannya di Internet atau media lain secara fulltext _{untuk kepentingan} akademis tanpa perlu meminta ijin dari saya selama tetap mencantumkan nama saya sebagai penulis/pencipta dan atau penerbit yang bersangkutan.

Saya bersedia untuk menanggung secara pribadi, tanpa melibatkan pihak Perpustakaan UIN Sunan Ampel Surabaya, segala bentuk tuntutan hukum yang timbul atas pelanggaran Hak Cipta dalam karya ilmiah saya ini.

Demikian pernyataan ini yang saya buat dengan sebenarnya.

Surabaya, 20 Februari 2017

Penulis

(Rizky Amalia Indayani)

KEMENTERIAN AGAMA

UNIVERSITAS ISLAM NEGERI SUNAN AMPEL SURABAYA

PERPUSTAKAAN

Jl. Jend. A. Yani 117 Surabaya 60237 Telp. 031-8431972 Fax.031-8413300 E-Mail: perpus@uinsby.ac.id

(6)

ABSTRACT

Indayani, Rizky Amalia (2017). Analytic Scoring Procedure in Speaking Assessment At Hospitality Program of State Vocational High School 1 Buduran Sidoarjo.

A Thesis. English Education Department, Faculty of Tarbiyah and Teacher Training, Sunan Ampel State Islamic University, Surabaya. Advisor: Dra. Arba’iyah Yusuf, M.A.

Key Words: Analytic Scoring Procedure, Analytic Rubric, Speaking

Students are categorized as a good or bad speakers depend on someone who assesses their ability while speaking. It is common that speaking assessment for classroom purposes done by the teachers has been done without any theoretical basis. So, the teachers just use their feel judgment to score the students’ performance. By doing that, the teachers determine the score based on their impression without making an analysis of the aspects of speaking ability. Thus, the research is aimed to figure out whether the teacher’s rubrics for speaking assessment of hospitality students at State Vocational High School 1 Buduran comply with the characteristic of Analytic Scoring Procedure or not. The subjects of this research are the teacher and the students of tenth grade students of Hospitality Program at State Vocational High School 1 Buduran. There are 2 rubrics used by the teacher. The researcher used descriptive qualitative method to present the data. Here, the researcher collected the data used three techniques, those are; observation checklist, interview guideline, and documents. The data was obtained from document and interview. After collecting the documents, the researcher analyzed it by using checklist rubric about the valid analytic scoring rubric for oral assessment adapted from two theories. First, the theory from Mary J. Allen in her book Developing and Applying Rubric,1 and the Theory from Susan M. Brookhartin his book by the title “How to Create and Use Rubrics for Formative Assessment and Grading”. In conclusion, these rubrics are compliant enough with the characteristics of analytical rubric based on Allen’s theory in both created process and the natures. In particular, rubric for presentation is compliant by having 70% compliant to the characteristic of analytic rubric based on the created process and 70% compliant based on the natures. Additionally, rubric for conversation based on the created process characteristic is compliant by having 70% compliance and 100% compliant based on the natures.

O'Malley, J.Michael, Authentic Assessment for English Language Learners. (London: Longmans, 1996).

(7)

ABSTRAK

Indayani, Rizky Amalia (2017). Prosedur Penilaian Analitik di Penilaian Kemampuan Berbicara Pada Program Perhotelan Negeri SMK 1 Buduran Sidoarjo. Sebuah tesis. Pendidikan Bahasa Inggris, Fakultas Tarbiyah dan Keguruan, Sunan Ampel Universitas Islam Negeri, Surabaya. Pembimbing: Dra. Arba'iyah Yusuf, M.A.

Kata kunci: Prosedur Penilaian Analitik, Analitik Rubrik, Berbicara

Siswa dikategorikan sebagai pembicara yang baik atau buruk tergantung pada seseorang yang menilai kemampuan mereka saat berbicara. Pada umumnya, Penilaian kemampuan berbicara untuk tujuan kelas yang dilakukan oleh guru telah dilakukan tanpa dasar teoritis. Jadi, para guru hanya menggunakan pertimbangan terhadap penampilan siswa pada saat berbicara. Dengan melakukan itu, para guru menentukan skor berdasarkan kesan mereka tanpa membuat analisis aspek kemampuan berbicara. Dengan demikian, penelitian ini bertujuan untuk mengetahui apakah rubrik guru untuk berbicara penilaian siswa perhotelan di SMK Negeri Tinggi 1 Buduran sesuai dengan karakteristik dari Prosedur Penilaian Analitik atau tidak. Subyek penelitian ini adalah guru dan siswa dari siswa kelas X Program Perhotelan di SMK Negeri Tinggi 1 Buduran. Ada 2 rubrik digunakan oleh guru. peneliti menggunakan metode deskriptif kualitatif untuk menyajikan data. Di sini, peneliti mengumpulkan data menggunakan tiga teknik, yaitu; ceklis observasi, wawancara pedoman, dan dokumen. Data tersebut diperoleh dari dokumen dan wawancara. Setelah mengumpulkan dokumen, peneliti menganalisis dengan menggunakan rubrik tentang rubrik analitik scoring berlaku untuk penilaian lisan diadaptasi dari dua teori. Pertama, teori dari Mary J. Allen dalam bukunya ‘Developing and Applying Rubric’2

, dan Teori dari Susan M. Brookhartin bukunya dengan judul ’How to Create and Use Rubrics for Formative Assessment and Grading’. Kesimpulannya, rubrik ini cukup

sesuai dengan karakteristik rubrik analitis berdasarkan teori Allen di kedua proses pembuatan dan sifat. Secara khusus, rubrik untuk presentasi sesuai dengan memiliki tingkat kesesuaian 70% dengan karakteristik dari rubrik analitik berdasarkan proses pembuatannya dan 70% sesuai berdasarkan cirinya. Selain itu, rubrik untuk percakapan berdasarkan karakteristik proses pembuatannya sesuai dengan memilikikesesuaian 70% terhadap proses pembuatan dan 100% sesuai berdasarkan sifat.

O'Malley, J.Michael, Authentic Assessment for English Language Learners. (London: Longmans, 1996).

(8)

TITLE SHEET ... i

ADVISOR APPROVAL SHEET ... ii

APPROVAL SHEET ... iii

MOTTO ... iv

DEDICATION SHEET ... v

ACKNOWLEDGEMENTS ... vi

ABSTRACT ... vii

PERNYATAAN KEASLIAN TULISAN ... ix

TABLE OF CONTENTS ... x

LIST OF TABLE ... xii

LIST OF APPENDIX ... xiii

CHAPTER I : INTRODUCTION A. Background of The Study ... 1

B. Statement of The Research Problem ... 5

C. Objective of The study ... 5

D. Significance of The Study ... 6

E. Scope and Limit ... 7

F. Key Terms ... 8

CHAPTER II : REVIEW OF RELATED LITERATURE A. Scoring Procedure of Speaking Test ... 9

B. Analytic Scoring Procedures ... 10

1. Advantages and Disadvantages of Analytic Scoring ... 11

2. Analytic Oral Language Scoring Rubric ... 13

3. Characteristics of Analytical Scoring Procedure ... 15

C. English for Specific Purposes (ESP) ... 24

D. Previous Study ... 25

CHAPTER III : RESEARCH METHODS A. Research Design ... 29

B. Subject of Research ... 30

C. Setting of Research ... 30

D. Data and Source of Data ... 31

E. Research Instruments ... 33

F. Data Collection Technique ... 34

(9)

H. Checking Validity of Findings ... 35 I. Research Stages ... 36

CHAPTER IV : FINDINGS AND DISCUSSION

A. Research Findings ... 39 B. Research Discussion ... 69

CHAPTER V : CONCLUSION

A. Conclusion ... 86 B. Suggestion ... 89

REFERENCES APENDICES

(10)

LIST OF TABLES

4.1. Characteristic of Analytic Rubric based on Created Process ... 41 4.2. Characteristic of Analytic Rubric ”based on the standard” ... 42 4.3. Characteristic of Analytic Rubric “Design to assess created authentic task” ... 43 4.4. Characteristic of Analytic Rubric “The number of criteria is the essential

elements of the task” ... 45 4.5 Characteristic of Analytic Rubric Everything on every task is not always assessed ... 46 4.6 Characteristic of Analytic Rubric Numbers of criteria are based on the kinds of assignment ... 47 4.7 Characteristic of Analytic Rubric The same number in levels of performance for each criterion within a rubric ... 48 4.8 Characteristic of Analytic Rubric Additional descriptors to each level of

performance ... 50 4.9 Characteristic of Analytic Rubric Assigning point values to performance on each criterion ... 52 4.10 Characteristic of Analytic Rubric Equal intervals between the point values in a rubric ... 53 4.11 Characteristic of Analytic Rubric Checking the Rubric ... 54 4.12 Characteristic of Analytic Rubric”based on the natures” ... 56 4.13. Characteristic of Analytic Rubric “Helps to examine efficiently complex products or behaviors” ... 57 4.14. Characteristic of Analytic Rubric “Helps to precisely define targets or

expectations”

(11)

4.21 Characteristic of Analytic Rubric Adaptable for summative assessment ... 66 4.22 Characteristic of Analytic Rubric Can be combined when overall score for grading is needed ... 67

(12)

LIST OF APENDICES

Appendix 1 Teacher’s rubrics

Appendix 2 Form of observation checklist

Appendix 3 Form of interview guideline

Appendix 4 Result ofobservation Checklist

(13)

CHAPTER I INTRODUCTION

This study is about analytic scoring procedure in speaking assessment at hospitality program of state vocation high school 1 buduran sidoarjo. This chapter discusses the area of the study that be covered in some headings; background of study, research questions of the study, objectives of the study, significance, scope and limitation, then definition of the key terms.

A.Background of The Study

Speaking seems intuitively the most important of all the four language skills (listening, writing, reading, and speaking) since people who know a language are referred to as a ‘speaker’ of that language, as if speaking included all other skill knowing that language1

. Through speaking, somebody shows their idea or their feeling. Moreover in English class, the teacher will know students idea through speaking. As another skill, the teacher conduct speaking test to measure that students understand and have a progress during study in the class.

A speaking test is a procedure to measure speaking ability that requires the learner to speak or to produce utterances and he/she is assessed

Mukminatien, Nur. 2000. The Advantages of Using an Analytic Scoring Procedure in Speaking Assessment, TEFLIN Journal, Vol. XI, No. 1 Agustus 2000. Universitas Negeri Malang.

(14)

on the basis of his/her utterances2

. The learners will know their progress, strengths and weakness during study the lesson in the class. Furthermore, through speaking test the teacher knows their students about their progress.

In an elementary level, a speaking test may be focused on assessing one or two aspects of speaking skills such as the pronunciation, intonation, and stress. In advanced level, the assessment is focused on the language function. Unfortunately, when speaking assessment is concerned, not all teacher when evaluates students using a detail description about how to develop a good scoring guide or a scoring sheet. Experts in language testing have discussed types and techniques of language testing without giving information on how to score the test3

Presently, learners are demanded to speak English during their English Lesson or in full case for English for Specific Purposes Students such students of Vocational High School in a hospitality program. They are forced to speak English fluently because the purpose of their study is getting in touch with many foreigners in their working world.

Underhiil, N. Testing spoken language. Cambririge: (Cambridge University Press:1987) p.98

(15)

judgment to score the students’ performance. By doing that, the teachers determine the score based on their impression without making an analysis of the aspects of speaking ability. Thus, the researcher directly shows the use of analytic scoring procedure in assessing speaking ability of students used English for Specific Purposes. Analytic schemes have also been found to be particularly useful for scorers who are relatively inexperienced4

Most of researchers find out the effectiveness of those scoring procedure by comparing both of analytic and impressionistic. In this research, the researcher wants to find out resemblance of the teacher’s rubric for speaking assessment with the characteristic of Analytic Scoring rubric and make it related with English for Specific Purposes (ESP). It should be analytic because this scoring procedure is more effective than the holistic one5

. There are several studies which is similar to this study. The first study is by Luu Trong Tuan, 2012, University for Natural Resources and Environment, Ho Chi Minh City, Vietnam, Teaching and Assessing Speaking Performance trough Analytic Scoring Approach. The aim of this research was to examine if the analytic scoring approach in teaching and assessment would be more effective for improving students’ speaking performance6

. The

Underhiil, N. Testing spoken language. Cambririge: (Cambridge University Press:1987) p.138

Mukminatien, Nur. 2000. The Advantages of Using an Analytic Scoring Procedure in Speaking Assessment, TEFLIN Journal, Vol. XI, No. 1 Agustus 2000. Universitas Negeri Malang.

Luu Trong Tuan. Teaching and Assessing Speaking Performance through Analytic Scoring Approach. University for Natural Resources and Environment, Ho Chi Minh City, Vietnam. Theory and Practice in Language Studies, (Vol. 2, No. 4, pp. 673-679, April 2012)

(16)

research also explored the students’ attitude towards the implementation of analytic scoring approach in speaking teaching and assessment.

The second study is Ying-Yung Chuang, Cheng Shiu University, China Taiwan. Foreign Language Speaking Assessment: Chinese Taiwanese College English Teachers’ Scoring Performance in the Holistic and Analytic Rating Methods. The purpose of this study was to investigate college English teachers’ scoring performance of the Holistic and Analytic rating methods, their views and concerns with the components of oral skills, and whether teachers’ background variables influenced their scoring performance.

This research only focuses on the teachers’ rubric in assessing speaking of hospitality students at Vocational High School 1 Buduran Sidoarjo. This school was chosen as the research place because the quality of the school. Some of big institutions whether domestic or foreign have cooperation with this Vocational High School. Relating the vision of SMKN 1 Buduran that is become an education and training institution which is able to produce professional, independent and competing graduates in national and International social work7

. In particular, Hospitality Program there is student exchange program which some of students has a chance to go overseas to job training. The program collaborates with Malaysia. thus, the students of hospitality program must speak English very well.

Profil SMKN 1 Buduran Sidoarjo. Accessed on http://www.smkn1buduran.sch.id/profil.html October, 20th 2016

(17)

B. Statement of the Research Problem

Based on the background of the study, the researcher analyzed the compliance of the teacher’s rubric for speaking assessment with the characteristic of Analytic Scoring rubric. In line with the statements described above, the researcher formulated the problems as follow:

Do the teacher’s rubrics for speaking assessment of hospitality students at State Vocational High School 1 Buduran comply with the characteristic of Analytic scoring procedure?

C. Objectives of the Study

Related to the problems above, the researcher has formulated the major objective of this study to describe the resemblance of teacher’s rubric for speaking assessment for English for Specific Program in terms of;

To figure out whether the teacher’s rubrics for speaking assessment of hospitality students at State Vocational High School 1 Buduran comply with the characteristic of Analytic Scoring Procedure or not.

(18)

D. Significance of Study

By completing this study, the researcher expected which brought much benefit to many people. The analytic procedure of speaking assessment provided a double function: as a valid and reliable instrument and as a diagnostic procedure or remedial teaching. Therefore, it serves as feedback both for the students and teacher.

1. For the Teacher

By doing an analysis in teachers’ scoring rubric used in assessing speaking, we can give more suggestion whether the rubric has already complied with the characteristic of analytic procedure or not. So, the teacher’s assessment for hospitality students will be more effective and the teacher can further make a decision on which speaking skill needs more practice in the remedial teaching.

2. For Business Travel Students

By doing this research and knowing that the procedure of scoring is based on the analysis of features, this scoring procedure can be used as feedback for the learner to improve his/her performance. So, after knowing the results of each student, they will know which language component that need to be improved.

(19)

E. Scope and Limit of the Study

The scope of this study is English teaching. Specifically, the main data is taken from the rubrics for assessing speaking in Individual presentation with expression of intention as the material and short conversation with expression congratulating others as the material. The elements and the characteristics of the rubrics are analyzed in both created process and the natures of an analytic rubric.

Furthermore, the study is limited to the teacher who has created and used the rubric for assessing students’ speaking skill. He is an English teacher for seventh grade of Hospitality students at State Vocational High School 1 Buduran.

F. Key Terms 1. Hospitality

Based on Cambridge Advance Learner Dictionary, hospitality is defined as a term when people are friendly and welcoming to guests and visitors. So, hospitality students are only a group of people at State Vocational High School 1 Buduran Sidoarjo who learn particular topics for hospitality where the English is needed to be used in certain situation especially for dealing with hotels’ customers.

(20)

2. Speaking

Speaking is an interactive process of constructing meaning which is comprised of producing and receiving information (Brown, 1994; Burns & Joyce, 1997). Besides, speaking also classified to monologue and dialogue8

. In education world, speaking practice is not just producing some words in specific purposes. But, it must be assessed so both of the teacher and learner will know someone’s ability in speaking.

3. Analytic Scoring Procedure

The procedure of scoring based on analysis of features is called analytic (Lyoyd-Jones, 1997; White, 1985; Spandel and Stiggins, 1990). This procedure provides two advantages especially for classroom purposes: as an instrument to measure the students’ speaking proficiency and as a diagnostic procedure for remedial teaching.

Nunan, D. Designing Tasks for the Communicative Classroom. New York: (Cambridge University Press:1989) p.87

(21)

CHAPTER II

REVIEW OF RELATED LITERATURE

In this chapter, the researcher attempts to provide two sub-sections. The first is the review of related literature which describes the basic theories used in this research. The second is review of previous study; explains the differences of this study from others which are conducted by other researchers.

A. Scoring Procedure of Speaking Test

The scoring procedure of a functional speaking test on the contrary is more complicated as it requires the rater to pay attention to the overall components of speaking skill at one time.1

The functional speaking test can be done in an interactive communication or transactive one (Brown and Yule in Mukminatien, 2000: 39). Interactive communication is referring to the function of language to maintain social interaction like interview and role play. The transitive interaction is referring to a type of communication that is focused on conveying the massage, and not on the interaction, like storytelling, giving speech, reading an announcement, presenting a report, and many others. 2

According to Underhill in Nur Mukminatien (2000: 39) there are two approaches in the scoring procedures of a speaking test, the analytic approach

Mukminatien, Nur. 2000. The Advantages of Using an Analytic Scoring Procedure in Speaking Assessment, TEFLIN Journal, Vol. XI, No. 1 Agustus 2000. Universitas Negeri Malang.

Pandiya, 2013. Rubrics on Scoring English Tests for Four Language Skills. Jurnal Pengembangan Humaniora Vol. 13 No. 1, April. Polines

(22)

and the impressionistic approach3. First is scoring the learners’ speaking ability

by separating the components of speaking skill into sub skills, and the rater scores each component, and then sums the sub scores into final score. The latter is scoring/judging the learners’ speaking ability on the basis of the rater’s general impression on the learners’ performance without necessarily separating the speaking components.

B. Analytic Scoring Procedures

Jones et.al, defines that the procedure of scoring based on analysis of features is called analytic. Trong Tuan also states that analytic scoring is assessment which the objectives of the final product is broken down into criteria parts, and each part is scored independently.4

The procedures of this method involve the separation of the various features of a discourse into categories for scoring purpose, meanwhile the final score from this analytic is the rating which is calculated and summarized from the total score for all of the parts that are being evaluated. Therefore, to avoid bias towards the whole product, it is necessary for the teacher to treat each criterion or part as separate. Related to this research conducting in analyzing an analytic scoring procedure that the teacher used in assessing speaking in Hospitality at State Vocational High School 1

Ibid..

Tuan, Luu Trong. 2012. Teaching and Assessing Speaking Performance through Analytic Scoring Approach . University for Natural Resources and Environment, Ho Chi Minh City, Vietnam. Theory and Practice in Language Studies, Vol. 2, No. 4, pp. 673-679, April 2012

(23)

Buduran, there are the advantages and the disadvantages of scoring procedures. It is described as follow:

1. Advantages and Disadvantages of Analytic Scoring procedures

a. The Advantages

Analytic scoring is preferred over holistic schemes by many speaking specialists for a number of reasons.5 It means that analytic scoring provides more useful diagnostic information about students’ speaking abilities in term of where their weaknesses are and where their strengths are. Downing and Haladyna (2006) argues that analytic scoring has been considered as more interpretable scoring approach because it can access the examinee’s specific strengths and weaknesses and can identify the particular components of speaking discourse that an examinee needs developing.

Park (2004) identified that the explicitness of the analytic scoring guides offers teachers a potentially valuable tool for providing speakers with consistent and direct feedback6. He also adds that analytic scoring are particularly useful for second language learners. Moreover, another author examines that some learners who acquire English as their second language may probably have excellent speaking skill in terms of its content and

Park, T. (2004). An Overview of Portfolio-based Writing Assessment. Accessed on April, 6th 2016. http://www.tc.columbia.edu/academic/tesol/WJFiles/pdf/TaejoonParkForum.pdf.

(24)

organization, otherwise they may have much lower grammatical control; others may have an excellent control of sentence structure, but may not know how to organize their speech in a logical way. Thus, the analytic scoring scales take a role when the same rubric categories repeatedly are used to show students’ knowledge if they have made progress in some or all aspects that being assessed7.

b. The Disadvantages

Another major of analytic scoring is the disadvantages of it. In this case, rating speaking performance takes a lot of time since examiners are required to make more than one decision for every speaking performance8. When scoring analytically, an examiner has to check, consider, and score each criterion of the speaking ability and then gives a total score depending on the coefficient put forward. Hughes (2003) warned that in scoring analytically, the criterion scored first may effect on subsequent criteria which are scored later, making the overall effect of a speech diverted to an individual criterion.

An additional problem with some analytic scoring schemes is that even experienced essay judges sometimes find it difficult to assign

Moskal, B.M. (2000). "Scoring rubrics: What, when and how?" Practical Assessment, Research & Evaluation, 7 (3). Accessed on http://pareonline.net/getvn.asp?v=7&n=3

Tuan, Luu Trong. 2012. Teaching and Assessing Speaking Performance through Analytic Scoring Approach.

(25)

numerical scores based on certain descriptors9. In this aspect, there are possibilities for scorers to disagree with one another. It is more difficult to achieve intra- and inter-rater reliability on all of the dimensions in an analytic scoring scheme than on a single score yielded by a holistic scale. This disadvantage is inevitable, especially with trained or un-experienced scorers. Grammar-related categories are somewhat wrong – right categories whereas other categories are judgments. Focusing on wrong – right categories will always be easier than judgments.

2. Analytic Oral Language Scoring Rubric

A rubric is a coherent set of criteria for students’ work that includes descriptions of levels of performance quality on some particular criteria. The main purpose of rubrics is to assess performances. Some performance performances that might assess with rubrics are Oral communications including Making a speech to the class, reading aloud, conversing in a foreign language.

The genius of rubrics is that they are descriptive and not evaluative. Of course, rubrics can be used to evaluate, but the operating principle is you match the performance to the description rather than “judge” it. Thus good rubrics have to State standards, curriculum goals, and instructional goals and objectives are the sources for what types of performances your students

Hamp-Lyons, L. (1989). Second language writing: Assessment issues. In Kroll, B. (Ed.), Second language writing (pp. 69-87). Cambridge: Cambridge University Press.

(26)

should be able to do. Rubrics are usually categorized by two different aspects of their composition. One is whether the rubric treats the criteria one at a time or together. The other is whether the rubric is general or is task-specific and only applicable to one assessment.

There are two approaches in the scoring procedures of a speaking test, the analytic approach and the impressionistic approach10. Analytic rubrics describe work on each criterion separately. Holistic rubrics describe the work by applying all the criteria at the same time and enabling an overall judgment about the quality of the work.

Indonesian teachers mostly tend to use numeric scores in interpreting students’ score. Even though particular measurements and the details of scoring method are included, the levels of the students’ achievement during the learning process and the explanation of their result are significantly needed. In connecting to this, some specific scales in different criteria and levels are considered. Therefore, most classroom purposes, analytic rubrics are best11. Focusing on the criteria because students can see what aspects of their work need what kind of attention. It will also be used to make decisions about the future. For example, it is a diagnostic procedure for remedial teaching and as a procedure to teach something next year.

Mukminatien, Nur. 2000. The Advantages of Using an Analytic Scoring Procedure in Speaking Assessment, TEFLIN Journal, Vol. XI, No. 1 Agustus 2000. Universitas Negeri Malang.

Brookhart, Susan M.(2013) How to create and use rubrics for formative assessment and grading

(27)

3. Characteristics of Analytical Scoring Procedure

a. Created Process

Rubrics provide the criteria for assessing students' work. They can be used to classify virtually any product or behavior, such as essays, research reports, portfolios, and works of art, recitals, oral presentations, performances, and group activities. Judgments can be self-assessments by students; or judgments can be made by others, such as faculty, other students, fieldwork supervisors, and external reviewers. Rubrics can be used to provide formative feedback to students, to grade students, and to assess courses or programs.

As mentioned above, rubric is divided into two kinds: holistic rubric (one global, holistic score for a product or behavior) and analytic rubric (separate, holistic scoring of specified characteristics of a product or behavior). Specifically, as this research is focused on analytic rubric, there are some particular characteristics explained by Allen in these following details12:

1) Based on the standards

Standards are typically one-sentence statements of what students should know and be able to do at a certain point. There are many sources that teachers can find examples of goals and standards that

(28)

might be appropriate for students. There are national and state standards as well as numerous websites with many good choices. Regarding to this, there are three steps in deciding the standards: reflect, review and write. Basically, it is unnecessary to start from scratch, which can confine teacher’s thinking. However, it is necessary to look at the work of others; therefore, considering what do teacher really wants the students to know and be able to do when they leave the grade or school. This is called as reflection. As a result of this reflection, teacher might reach consensus on a few things you most value and agree should be included in the standards. Teacher might actually write a few standards or, teacher might produce a long list of possible candidates for standards. Furthermore, reviewing is the next step. Then, the last one is writing. After writing the standards, it is considerable to make guiding questions to the standards makers or teachers’ partners.

2) Designed to assess created authentic task

Authentic assessment does not compete with traditional assessments like tests. Rather, they complement each other. Each typically serves different assessment needs, so a combination of the two is often appropriate. As teacher wants to find a way students can demonstrate that they are fully capable of meeting the standard. It is necessary to

(29)

consider the authentic task is also designed based on the standards and the real world. Therefore, as the task for students is good, the analytic rubric should be created and scaled based on the task given as well. 3) Consists of the characteristics of good criterion

Criteria here mean indicators of good performance on a task. In order to concept an analytic rubric, questions such as "What does good performance on this task look like?" or "How will I know they have done a good job on this task?" frequently need to be appeared. In answering those questions teacher will be identifying the criteria for good performance on that task. Teacher will use those criteria to evaluate how well students completed the task and, thus, how well they have met the standard or standards. Regarding to this, the first thing to be considered is the number of criteria is the essential elements of the task; the next one is everything on every task is not assessed; and the last is numbers of criteria are based on the kinds of assignment.

4) The same number of levels of performance for each criterion within a rubric

As mentioned before, keep the number of criteria manageable. Teacher does not have to look for everything on every assessment. Teacher could have five levels of performance for three criteria in a

(30)

rubric, three levels for two other criteria, and four levels for another criterion, all within the same rubric. Rubrics are very flexible. There is no need to force an unnatural judgment of performance just to maintain standardization within the rubric. If one criterion is a simple either/or judgment and another criterion requires finer distinctions, then the rubric can reflect that variation.

5) Additional descriptors to each level of performance

Descriptors are recommended but not required in a rubric. As described in rubrics, descriptors are the characteristics of behavior associated with specific levels of performance for specific criteria. For example, in the following portion of an elementary science rubric, the criteria are observations are thorough, predictions are reasonable, and conclusions are based on observations. Labels (limited, acceptable, proficient) for the different levels of performance are also included. Students will be more certain what are expected to reach each level of performance on the rubric if descriptors are provided. Furthermore, the more detail a teacher provides about what good performance looks like on a task the better a student can approach the task. Teachers benefit as well when descriptors are included. A teacher is likely to be more objective and consistent when applying a descriptor such as "most observations are clear and detailed" than when applying a simple label

(31)

such as "acceptable." Similarly, if more than one teacher is using the same rubric, the specificity of the descriptors increases the chances that multiple teachers will apply the rubric in a similar manner. When a rubric is applied more consistently and objectively it will lead to greater reliability and validity in the results.

6) Assigning point values to performance on each criterion

As mentioned above, rubrics are very flexible tools. Just as the number of levels of performance can vary from criterion to criterion in an analytic rubric, points or value can be assigned to the rubric in a myriad of ways. For example, a teacher who creates a rubric might decide that certain criteria are more important to the overall performance on the task than other criteria. So, one or more criteria can be weighted more heavily when scoring the performance. For example, in a rubric for solo auditions, a teacher might consider five criteria: (how well students demonstrate) vocal tone, vocal technique, rhythm, diction and musicality. For this teacher, musicality might be the most important quality that she has stressed and is looking for in the audition. She might consider vocal technique to be less important than musicality but more important than the other criteria. So, she might give musicality and vocal technique more weight in her rubric. She can assign weights in different ways.

(32)

7) Equal intervals between the point values in a rubric

Analytic rubrics are flexible tools. Shape them to fit teacher’s needs, not the other way around. In other words, points should be distributed across the levels of a rubric to best capture the value you assign to each level of performance. So, feeling bound by any format constraints when constructing a rubric is not necessary. The rubric should best capture what you value in performance on the authentic task. The more accurately your rubric captures what you want your students to know and be able to do the more valid the scores will be.

8) Checking the rubric

As a final check on a rubric, teacher can do any or all of the following before applying it: let a colleague review it, let students review it -- is it clear to them, check if it aligns or matches up with standards, check if it is manageable, consider imaginary student performance on the rubric. The last suggestion, imagine that a student had met specific levels of performance on each criterion (for an analytic rubric). Then make sure if that performance translates into the score that is seemingly appropriate. Of course, teacher will never know if he/she really has a good rubric until it is applied. So, working to perfect the rubric before you administer it is not necessary. Get it in good shape

(33)

and then try it. Find out what needs to be modified and make the appropriate changes.

b. Natures

Scoring rubrics are explicit schemes for classifying products or behaviors into categories that vary along a continuum. Some particular natures are identified based on the advantages and the function of analytic rubric; it can be identified as these following points:

1) Helps to examine efficiently complex products or behaviors

Specific criteria in analytic rubric help teacher a lot in examining complex products created by students or behaviors that are going to be measured. The levels also give specific and clearer explanation for characteristics that have good or bad scores.

2) Helps to precisely define targets or expectations

The specific criteria mentioned in analytic rubric can facilitate teacher in indicating the definition of targeted or expected achievement for students’ products, performances, and behaviors that are going to be assessed.

3) Criterion-referenced

Each criterion of the criteria mentioned in the rubric has to have specific description in each scale; what kind of achievement is targeted

(34)

or expected. In addition, the description should be referenced based on the criterion.

4) Ratings can be done by students to assess their own work or others

In analytic rubric, as there are specific definition of the target and expectation, the scales are also clear enough for people who read it, so the assessment is possibly applicable for the students who has their works or even other students to assess their friends.

5) Gives diagnostic information to teacher

As this is talking about analytic rubric, the result of the assessment process using the rubric should be analytic as well. It means the information should be clear and detail based on the criteria and the explanation about the target that students have to achieve.

6) Gives formative feedback to students

As explained in the previous point, analytic rubric is accessible for everyone, both students and teacher to assess something. This means, the feedback that is reflected as a result of measurement using the rubric is clearer for everyone as well; moreover for students whose product to be assessed.

(35)

7) Easier to link to instruction

When teacher gives task to students, there must be instructions as well. Therefore, every criterion in the rubric has to be connected to the instruction from the teacher as well.

8) Accessible for formative assessment

According to Susan, analytic rubric is designed to assess formative assessment. The goal of formative assessment is to monitor student learning to provide on-going feedback that can be used by teacher to improve their teaching and by students to improve their learning. Specifically, it helps students to identify their strengths and weaknesses and target areas that need works.

9) Adaptable for summative assessment

As explained above, analytic rubric is created for formative assessment; as consequences, summative assessment can be measured by this rubric when it is adapted. The goal of summative assessment is to evaluate students learning at the end of an instructional unit by comparing it against some standards. It often has a high point value. 10)Can be combined when overall score for grading is needed

It is easier for teacher to wrap up the result of students’ achievement during the learning process or the result of the task by combining the scores based on the scales in each criterion. As long as the scales are

(36)

countable and it can be rated based on the grade, the analytic rubric is considerable.

C. English for Specific Purposes (ESP)

Over the last three decades, English for specific purposes (ESP) has become a successful field13

. English was born in the 190s as a technical texts-analysis area14

. English for Specific Purposes (ESP) has boomed in regular education such in Vocational High School. Some of vocational High School inserts ESP in learning English. In addition, the approach to language description for English Academic Purposes developed by Swales (1990) has been embraced by ESP15.

Based on Swales, 1992: 300 English for specific purposes (ESP) is a subdivision of a wider field, language for specific purposes which is defined as the area of inquiry and practice in the development of language programs for people who need a language to meet a predictable range of communicative needs16

.in addition, English for Specific Purposes (ESP) is a learner-centered approach to teaching English as an additional language which focuses on developing English communication skills in a specific discipline, such as accounting, management, education, law, engineering, IT technology, health and

Gonzales, Carolina Ramirez, 2015 English for Specific Purposes: Brief History and Definitions.

Costa Rica University. P 380

Ibid 381

Basturkmen, Helen. 2010. Developing Courses in English for Specific Purposes. University of Auckland: New Zealand) P. 10

Swales, John. (1992). Language for specific purposes. In W. Bright (Ed.), International encyclopedia of linguistics (Vol. 2, p. 300). New York, Oxford: Oxford University Press.

(37)

medicine. It involves teaching and learning the specific skill and language needed by particular learners for a particular purpose.

In other hand, English for Specific Purposes (ESP) as a learner-centered approach, its main purpose has been that of fulfilling the specific needs of target learners to satisfy either their professional or vocational demands17

. Typically, ESP has functioned to help language learners cope with the features of language or to develop the competencies needed to function in a discipline, profession, or workplace18

D. Previous Studies

It has been prove that Analytic scoring procedure more advantageous compared to the impressionistic one. But, some researchers have studied some issues dealing with Analytic Scoring procedure in different focus and different sub. The findings of studies almost show same result.

The first study is by Luu Trong Tuan, 2012, University for Natural Resources and Environment, Ho Chi Minh City, Vietnam, Teaching and Assessing Speaking Performance trough Analytic Scoring Approach. The aim of this research was to examine if the analytic scoring approach in teaching and assessment would be more effective for improving students’ speaking

Gonzales, Carolina Ramirez, 2015 English for Specific Purposes: Brief History and Definitions.

Costa Rica University. P 382

1818

Basturkmen, Helen. 2010. Developing Courses in English for Specific Purposes. University of Auckland: New Zealand) P. 6

(38)

performance19

. The research also explored the students’ attitude towards the implementation of analytic scoring approach in speaking teaching and assessment.

Then the second study is by Ying-Yung Chuang, Cheng Shiu University, China Taiwan. Foreign Language Speaking Assessment: Chinese Taiwanese College English Teachers’ Scoring Performance in the Holistic and Analytic Rating Methods. The purpose of this study was to investigate college English teachers’ scoring performance of the Holistic and Analytic rating methods, their views and concerns with the components of oral skills, and whether teachers’ background variables influenced their scoring performance. 20

The third study is by Daruwan Srikaew, Kamonwan Tangdhanakanond, Sirichai Kanjanawasee, Chulalongkorn University, Thailand. English Speaking Skills Assessment for Grade 6 Thai Students: An Application of Multivariate Genenralizability Theory. This research studies an analytic rating scale for an English speaking skills assessment designed for Grade 6 Thai students learning English as a foreign language. The results showed that fluency was the greatest variance component of the composite score of the analytic rating scale, followed by ideational function, cohesion, vocabulary and syntax, and pronunciation

Ying-Yung Chuang. Foreign Language Speaking Assessment: Chinese Taiwanese College English

Teachers’ Scoring Performance in the Holistic and Analytic Rating Method. (Vol.8 Issues 11) Cheng Shiu University, China Taiwan.

(39)

respectively. Reliability of the composite score for the speaking analytic rating scale was high. It was found that the reliability coefficients for each component would be reliable when six or more tasks are used and the number of raters is from 6–10 and above. The dependability increased more when the number of tasks increased and when the number of raters increased. It was also found that a reliable high Phi Coefficient (over .80) could be obtained using only 6 tasks and 3 raters21

The forth study is by Chunguang Tian, Binzhou University. On the Construct Validity of an Analytic Rating Scale for Speaking Assessment. This paper aims to study the construct validity of an analytic rating scale for speaking assessment. The Multi-facet Rasch Modeling method and the correlations analysis are combined to investigate the construct validity. the analytic rating scale can reflect the students’ speaking ability22

The fifth study is by Cynthia S. Wiseman, City University of New York. A Comparison of the Performance of Analytic vs. Holistic Scoring Rubrics to Assess L2 Writing. This study compared the performance of a holistic and an analytic scoring rubric to assess ESL writing for placement and diagnostic purposes in a community college basic skills program. The study also

Daruwan Srikaew, Kamonwan Tangdhanakanond, Sirichai Kanjanawasee, Chulalongkorn University, Thailand. English Speaking Skills Assessment for Grade 6 Thai Students: An Application of Multivariate Genenralizability Theory. (2015, Vol 16, 47–66 p).

Chunguang Tian, Binzhou University. On the Construct Validity of an Analytic Rating Scale for Speaking Assessment. (International Journal of English Language Teaching Vol. 4, No. 1; 2017)

(40)

investigated whether each category in the two six-point rubrics were useful. Both scales appeared to be measuring a single latent trait of writing ability23

The last study is by Nur Mukminatien, State University of Malang. The Advantages of Using Analytic Scoring Procedure in Speaking Assessment. This article intends to describe the advantages of Analytic Scoring Procedure and to make a proof that Analytic scoring procedure is more advantageous compared to the impressionistic (holistic) one. Besides directed to show that using an analytic procedure in speaking assessment, in which the scoring procedure is based on the analysis of features of speaking ability, is more advantageous than using an impressionistic one (sometimes called a holistic approach) for two major reasons.

The fifth study is by Cynthia S. Wiseman, City University of New York. A Comparison of the Performance of Analytic vs. Holistic Scoring Rubrics to Assess L2 Writing. (Iranian Journal of Language Testing, Vol. 2, No. 1, March 2012)

(41)

CHAPTER III RESEARCH METHOD

This chapter deals with the procedures for conducting the study. It covers Research design, subject of research, data and source of data, research process, research instrument, data collection technique, and data analysis technique.

A. Research Design

This study designed to obtain information and description concerning teachers’ rubric for speaking assessment then found out whether the teacher’s rubric resemble to the characteristic of Analytic Scoring procedure or did not. Therefore, the design of this study was descriptive-qualitative since the purpose of this study is to describe the resemblance of a rubric. This descriptive study used to gain certain information about a certain phenomenon that happens when this study is conducted1

Based on Wallace, J, qualitative research is a method that uses to analyze the data which cannot be counted. This research was coded the data based on explanation of informants. Moreover, qualitative research was concerned with developing explanations of social phenomena2

Bogdan and Biklen. Qualitative Research for Education. P 11

Beverley Han cock, Elizabeth Ockleford, Kate Windridge, An Introduction To Qualitative Research, national institute for health research, 7, accessed: http://www.rds-yh.nihr.ac.uk , on: 31-03-2016.

(42)

B. Subject of Research

The research subject of this research is the English teacher of hospitality program in SMKN 1 Buduran, Sidoarjo. He is Mr. Mochammad Anwar, which is only English teacher of hospitality program.

Mr. Anwar finished is graduated from English Education Department of STKIP PGRI Sidoarjo in 1999. He started to teach since 2000/2001. During his career, he had teaching experiences in three different schools. First, he taught at SMA Tribhakti Tanggulangin from the academic year 2000/2001 to 2009/2010. Then, he started to teach English in MA Mambaul Hikam Tanggulangin from the academic year 2002/2003 to 2009/2010. At the same year, he also started to teach English in SMKN 1 Buduran. In 2010, he decided to resign from two previous schools before and focused on teaching English in hospitality program at State Vocational High School 1 Buduran.

C. Setting of the Research

1. Place

The research was conducted at SMKN 1 Buduran which was located in St. Jenggolo No. 1B Sidoarjo. The research was focused on Hospitality program because this is a program where the students have more chance to practice their English in the real communication related to

(43)

their future career3. Therefore, there would be more chance for the teacher to use authentic materials in English language teaching.

2. Time

The research was done starting from 6th June 2016. Document analysis was started from 6th of October until a week before the last interview. The interview was done on 11th June 2016. To complete the missing data, the second interview was done on 29th of October 2016. By conducting the interview after school hour and on the weekend, it would not disturb the teaching and learning process.

D. Data and Source of Data

1. Data

The data that is used in this study is elements of the rubric used by the teacher in assessing students’ speaking skill for two lessons and materials. Particularly, the first rubric is a scoring procedure for individual presentation which is used for ‘expression of intention’ lessons. The second rubric is used for assessing short conversation and the material is about ‘congratulating others’. In addition, teacher’s responds towards interview given by the researcher will support the data got from the rubric’s observation checklist.

Interview with English teacher of Hospitality program in SMKN 1 Buduran on Monday, 28th March 2016

(44)

2. Source of Data

Related to the data that are needed in this research, as the document is the teacher’s rubrics; the documents is obtained the teacher’s file. Also, the respondent of the interview of this study is the teacher as well. Therefore, the source of the data here is the English teacher for seventh grade of Hospitality students at State Vocational High School 1 Buduran, Sidoarjo.

E. Research Instruments

1. The main instrument

The main instrument in this research was the researcher. The researcher collected the data from Document study (Rubric Checklist) and teachers. The researcher analyzed the rubric and interview from the teacher.

2. Instrument Tools

There were two instrument tools that were used in this research:

a. Interview guidelines are questions for the subject of this research and this is often used to obtain the subject perception and attitudes of the issues. The methods are highly structured, formal interview, taped, and transcribed. In this researcher, the researcher used interview style semi-structured. From this style the researcher developed the ideas from the subject.

(45)

Analyzing the interview data used two key ways. The first is thematic generation is describe the perspective of the subject and make the conclusion. The second is citation is directly quote the subject’s sentences. In this case the researcher should directly write the name of the subject4

. The researcher used both of two ways to answer the research questions.

b. Observation Checklist

The observation checklist is needed to analyze the rubrics. The document needed to be analyzed in the teacher’s rubric for speaking assessment of hospitality students at State Vocational High School 1 Buduran. To analyze the rubric, the result of interview with the teacher was used. The result of both Checklist rubric and interview answered the research question.

c. Document

Document study was the record or history of the data that the research analyzed. The document in this study is the teacher’s rubric for assessing speaking in tenth grade of hospitality students at State Vocational High School 1 Buduran. To analyze the rubric, the result of interview with the teacher was used. The result of both Checklist rubric and interview answered the research question.

Stuart MacDonald & Nicola Headlam, Research Methods Handbook Introductory guide to research methods for social research (Manchester:CLES, 2014), P. 43.

(46)

d. Tape Recorder

This interview has been done orally. In interviewing the teacher, the researcher use recorder in order to keep the data. As the result of interview is used as a supporting data as the in answering the research question, it is very important to make sure that the data would not be lost.

F. Data Collection Technique

In this research, the data was collected from the rubric used by the teacher to assess students’ ability in speaking. The researcher asked the rubric and analyzed it by using checklist rubric; did an interview with the teacher’s to enrich understanding of the data. There are three kinds of interview, they are: formal, informal, and standardized interview5. This study used formal interview because the researcher had an interview with some of the teachers of Hospitality program at State Vocational High School 1 Buduran Sidoarjo. The result of both Checklist rubric and interview answered the research question.

G. Data Analysis Technique

The researcher analyzed the data by using descriptive qualitative method. The data was obtained from document and interview. After collecting

J. Amos Hatch, Doing Qualitative Research In Education Settings, State University Of Newyork press, 92, accessed: www.bookfi.org , on: 04-04-2014

(47)

the documents, the researcher analyzed it by using checklist rubric about the valid analytic scoring rubric for oral assessment adapted from two theories. First, the theory from Mary J. Allen in her book Developing and Applying Rubric,6

and the Theory from Susan M. Brookhartin his book by the title “How to Create and Use Rubrics for Formative Assessment and Grading”7. So, to analyze the information from the whole data, it would be conducted by using descriptive manner as follow8:

1. Identified the data based on the statement of problem

2. Classified each information based on each item of the interview and observation

3. Verified and interpret the data 4. Draw the conclusion

H. Checking Validity of Findings

In checking validity of the findings, the researcher used triangulation technique. As explained by Angen, triangulation in qualitative research is generally used as technique to ensure that data of the research is rich, robust,

O'Malley, J.Michael, Authentic Assessment for English Language Learners. (London: Longmans, 1996). P. 46

Brookhart, Susan M.(2013) How to create and use rubrics for formative assessment and grading (Alexandria, Virginia, USA)

Suharsimi Arikunto, Prosedur Penelitian Suatu Pendekatan Praktik. (Jakarta: Rineka Cipta, 2006), P. 152.

(48)

comprehensive, and well-developed.9 Additionally, Creswell also revealed that using a single method can never adequately shed light on a phenomenon; therefore multiple methods or techniques are needed in order to get deeper understanding.10

In Denzin, there are four types of triangulation; those are methods triangulation, triangulation of sources, analyst triangulation, and theory/perspective triangulation.11 When analyzing data, the researcher also crosschecked the findings using theory/perspective triangulation; as several Analytic scoring procedure theories were regarded. Furthermore, analyst triangulation was applied as well; besides the researcher, expert lecturers, including the advisors also have ensured the findings.

I. Research Stages

In this research stages, the research explained the process of conducting the research, including: preliminary research, planning, collecting data, analyzing, and concluding data, as these following explanations:

1. Preliminary research

For the preliminary research, the researcher made an appointment with the teacher at State Vocational High School 1 Buduran Sidoarjo to have

MJ Angen, Qualitative Health Research, vol. 10, 2000, p. 379,

John W. Creswell, Qualitative Inquiry and Research Design Choosing Among Five Traditions

(Thousand Oaks, CA: SAGE Publications Inc., 1998), p. 57

(49)

a consultation among this research. The researcher asked some questions about practicing speaking and the way to assess it to make sure that the teacher assesses it by using a rubric.

2. Planning

After conducting the preliminary research, the researcher made the instruments for collecting the data of the study. A checklist rubric was provided for assessing the compliance of teacher’s rubric with the theory of analytic scoring rubric. Those checklist rubric as the instrument of this study had to be validated first to the lecturer. After passing the process of instruments validity, the researcher conducted the research by using the instruments of research.

3. Collecting Data

In this stage, the researcher collected the data from a rubric used by APH’s teacher to assess their students’ speaking ability. Then, the teacher’s rubric was analyzed by using a document checklist to collect some data from it.

4. Analyzing

After collecting the data, the researcher analyzed the teaching stages of the lesson plans with the guidance of the theory in the literature review. The result of the analysis was written descriptively. The researcher analyzed the data based on the technique explained above. Then

(50)

answering the following research question and the result of the analysis were written quantitatively at the findings and discussions.

5. Concluding data

The research summarized all information which found by the researcher. The researcher served the data quantitatively. Then, the researcher discussed all information concerning the research to obtain the research data. Finally, the research concluded the research findings to answer the research questions.

(51)

CHAPTER IV

FINDING AND DISCUSSION

This chapter presents the collected data from teacher’s analytic scoring rubrics and teacher’s responds towards the interview, also the analysis of it. The results of the checklist also the teacher’s responds are showed as research findings. Furthermore, the analyzed data is explained in discussion part.

A. Research Findings

Following the research question that discuss about the teachers’ rubric compliance with the characteristic of analytic scoring procedure, the researcher serves the data that found at State Vocational High School 1 Buduran. This section contains on the detail information about the result of research which is used two kinds of instrument those are observation checklist and interview. The result of observation checklist presented in tabular form which is followed by the

result of teacher’s explanation supporting the checklist made by the researcher

which is taken from the interview section. The teacher’s explanation is presented descriptively.

In particular, the two analytic rubrics are used in this research. Both of those rubrics are rubric for assessing speaking skill of Vocational students especially for hospitality students. Based on teacher’s explanation, the rubrics are

(52)

40 made by team teaching of that school then distribute to each proper class. Based on the lesson plan which the researcher gained from the teacher, the first rubric is created for assessing students speaking skill on learning material expression of intention in individual presentation. The second rubric is for dialogue congratulating others material.

Although those rubrics are created by the same person, but each rubric has differences in some aspects based on created process and natures. From the checklist of analytic rubric criteria, both of the first rubric and the second rubric show several dissimilar results in some points of analytic rubric’s characteristics stated by Allen as explained in theoretical framework in chapter 2.

1. The Compliance Assessing Rubric with Analytic Scoring Procedure Based on Created Process

Compliance with effective analytic scoring procedure has particular indicators from two points of view. This section is focused on the created process’ point of view; this following table explains about the result of the observation checklist for the first rubric which assesses individual presentation and short conversation, followed by the percentage, those are:

(53)

41 Table 4.1

Characteristic of Analytic Rubric based on Created Process

The table above is an illustration about the result of the observation checklist. Specifically, related to the research question which tries to figure out the

compliance of the teacher’s rubric for assessing individual presentation and

rubric for assessing short conversation based on created process, these following tables represent explanation about the result of the analysis using the observation checklist and the interview.

(54)

42 Table 4.2

Characteristic of Analytic Rubric “Based on the Standard” Aspects Rubric

Created Process

Yes No

Teachers’ Process in

creating rubric

Characteristic s of creating

Analytic Rubric Based on the standards Rubric of Individual Presentation (Expression of Intention)

The teacher just refers to the material that would be delivered. There is no reflection, and reviewing process in creating it, means that those rubrics are not based on the standard made by teachers.

There are three steps in deciding the standards: reflecting (considering what do teacher really wants the students to know and be able to do when they leave the grade or school), reviewing (asking students) and writing (making rubric’s format) √

Rubric of Short Conversation (Congratulating Others)

The rubric for conversation is not based on the standard made by the teachers. Teachers should design the rubric by using the standart of teaching process. Properly, standards made by teacher or rubric developer resulted from reflection, reviewing and writing. Otherwise, the teacher is designing the analytic scoring rubric refers to the material that would be delivered.

√

The table above shows that both rubrics are not based on the standard, because the rubric developers did not make an official standard in creating the

rubric with the three steps based on the theory. In addition, the teacher’s respond

to the interview shows that there is no standard exactly in creating an assessment rubric for speaking test, but he always considers what kind of task that he will give to the student, is the rubric relates with the lesson in the class. Actually, the

(55)

government provides rubric for speaking, but the teacher didn’t adapt it, he

adopted it when he will use it. Furthermore, the ways the teacher created the rubric are: first, he decided what speaking that he will assess to my student. Have they learned about that, Then he made a rubric consists of criteria and aspect that he will assess. Finally, based on the interview, the teacher indeed did not make

the standard. He said that “There is no valid standard in creating assessment

rubric for speaking test only suggestion from government. I created assessment rubric by myself. I consider about students ability and the material that they

studied in creating assessment rubric.”1

It means that every rubric are created are

not based on some teachers’ thought, that related to students need and the real fact of some learning activities.

Table 4.3

Characteristic of Analytic Rubric “Design to Assess Created Authentic Task”

Aspects Rubric

Created Process

Yes No

Teachers’ Process in

creating rubric Characteristics of creating Analytic Rubric Design to assess created authentic task Rubric of Individual Presentation (Expression of Intention)

Used for assessing speaking in individual presentation. Means, rubric a are not designed to assess authentic task. Here, authentic task means assesses activities are natural activity.

Each typically serves different assessment needs, so there is no preparations, no settings of students when the teacher assesses their speaking skill. √

Rubric of Short Conversation (Congratulating

Teacher only assessed the materials that have been taught by the

√

(56)

Others) teacher. they have been prepared about

expression of

congratulating others and the activities are

dialogue in pairs, conversation and short role playing; which are the script are created by the students. Whereas, the teacher did not give specific explanation about where would

students’ use these skills

in the real world.

As teacher wants to find a way students can demonstrate that they are fully capable of meeting the standard

The table above shows that both of the rubrics are not designed to assess authentic task. When the teacher was interviewed “What are the considerations in creating assessment rubric for speaking?” his answer is ‘First thing the

consideration in creating assessment rubric is students’ ability. If they understand

the lesson, that make me easier in creating rubric. In particular, decide each aspect with the criteria. It should proper with students’ ability.’ Furthermore, for

the question ‘What kind of task/assignment did you give to your students? Did you give time to your students to prepare? Is the task related to the real world or applicable in their daily life?’2 his answer ‘was there are two kind of rubric; the first rubric is about expression of intension then individual presentation with holiday theme. Then the second rubric is about congratulating others with dialog in pair.’

(57)

45 Table 4.4

Characteristic of Analytic Rubric “The Number of Criteria is The Essential Elements of The Task”

Aspects Rubric

Created Process

Yes No

Teachers’ Process in

creating rubric Characteristics of creating Analytic Rubric The number of criteria is the essential elements of the task Rubric of Individual Presentation (Expression of Intention)

In this rubric, all of the criteria in the rubric are important because it contains scale. Every score of it contains percentage that explains the number of each

student’s ability. So, if

one of the number of criteria are missed, caused the rubric are not ideal anymore.

Limited the number of criteria; the rubric developer should keep it to the essential elements of the task. The criteria explained in each

achievement level are important and related to the task.

√

Rubric of Short Conversation (Congratulating Others)

From five aspect including five criteria, some of them do not have an important impact and significant differences in achievement levels. It shown at one of criteria in grammmar aspect. There is criterion that can be complied with another criterion.

√

It is clearly stated in the table above that the number of criteria is the

essential elements of the tasks in teacher’s rubric for assessing individual

presentation. Meanwhile, the criteria of rubric for assessing Short Conversation (Congratulating Others) are not the essential elements of the task. To enrich the

data, the teacher’s respond towards the question “What are the considerations in creating the criteria in each aspect of the rubric?”is “I myself consider some

(1)

formative assessment, adaptable for summative assessment, and can be combined when overall score for grading is needed.

3. In the created process’ point of view, the rubric for assessing short conversation is also not based on the standards, it is also not design to assess created authentic task, and not all of the number of criteria is the essential elements of the task. On the other hand, everything on every task in this rubric is not always assessed; numbers of criteria are based on the kinds of assignment; it has the same number in levels of performance for each criterion within a rubric, and additional descriptors to each level of performance. Additionally, it has assigning point values to performance on each criterion, equal intervals between the point values in a rubric, and checking the rubric requirement.

4. The natures of second rubric are completely compliant with the natures of effective analytic scoring rubric. It helps to examine efficiently complex products or behaviors and it helps to precisely define targets or expectations. The ratings can be done by students to assess their own work or others and it can give formative feedback to students. Moreover, it is criterion-referenced, gives diagnostic information to teacher, easier to link to instruction than holistic rubrics, accessible for formative assessment, adaptable for summative assessment, and can be combined when overall score for grading is needed.

(2)

B. Suggestion

The researcher proposes some suggestions to the students, teachers, and next researchers. They are as these following points:

1. To the English teacher

It will be better if the teacher makes a standart for creating analytic rubric of assessing speaking. The teacher should complete the teachers’ scoring rubric with the comprehensive explanation about the assessment score in speaking. Teacher should have some discussion agenda for creating the standart for teaching analytic rubric of assessment speaking. The teacher also

should have to give the students’ feed back from previous students’ activities by using teachers’ scoring rubric. Therefore,students can evaluate their own result from the scoring rubric in assessing speaking that complied with the characteristic of analytic procedure.

2. To the students

The students should know about the teachers’ scoring rubric that used by the teacher. The students can evaluate theirselves based on the result of

teachers’ scoring rubric. Therefore, students will know which language component that need to be improved. Not only for assessing their own result, the students also should use the teachers’ scoring rubric for their pair assessment. The students should give about students’ feedback for designing the standar of teacher’s rubric for speaking assessment.

(3)

3. To the next researcher

The researcher had known that the teacher’s rubric for speaking assessment has complied with the characteristic of analytic scoring procedure. Otherwise, the researcher found there are two approaches in the scoring procedures on speaking test. As mentioned in the scope and limit of the study, the researcher only focus on Analytic approach of the teacher’s rubric assessing speaking. Therefore, the next researcher should condact the research about the impressionistic approach of the teacher’s assessing speaking as onother approach of scoring procedure for speaking test which did not covered by the researcher in this research.

(4)

REFERENCES

Allen, I. E., & Seaman, J. (2005). Growing by degress: Online education in the United States, 2005. Needham , MA : Sloan-C. Retrieved December 4, 2005, from http://www.sloan-c.org/resources/growing_by_degrees.pdf

Allen, Mary J. 2014. Developing and Applying Rubrics. Miami: Dade College

Arikunto, Suharsini, “Prosedur Penelitian (Suatu Pendekatan Praktik)”, (Jakarta: Rineke Cipta, 2006), 152

Ary, Donald. Introduction to Research in Education, (USA:Wadsworth, 2010) p. 452 Bachman, L. F. & Palmer, A.S. (1996). Language Testing In Practice (second

edition). Oxford: Oxford University Press.

Basturkmen, Helen. 2010. Developing Courses in English for Specific Purposes. University of Auckland: New Zealand)

Benbunan-Fich, R., Hiltz, S. R., & Harasim, L. (2005). The online internation learning model: An integrated theoretical framework for learning networks. In S. R. Hiltz, R. G. Goldman (Eds.), Learning together online (pp. 20-37) Benson, A. (2002). Using online learning to meet workforce demand: A case study of

stakeholder inﬂuence. Quarterly Review of Distance Education, 3(4), 443-452 Beverley Han cock, Elizabeth Ockleford, Kate Windridge, An Introduction To Qualitative Research, national institute for health research, accessed: http://www.rds-yh.nihr.ac.uk, on: 02-05-2014.

Brown, H.D. 1987. Principles of language learning and Teaching. New Jersey: Prentice Hall.

Chyung, S. Y., & Vachon, M. (2005). An investigation of the satisfying and dissatisfying 113 factors in e-learning. Performance Improvement Quarterly, 18, 97-114.

Clyde, William and Delohery, Andrew, The guide to classroom technology for mainstream faculty (New Haven: Yale University Press, 2004). 78

Cunningsworth, Alan, “Choosing Your Coursebook” (United KIngdon: Macmillan Heinemann, 1998)

(5)

Gonzales, Carolina Ramirez, 2015 English for Specific Purposes: Brief History and Definitions. Costa Rica University.

H, Amosh, 2002, Doing Qualitative Research In Education Setting, State University of New York Press, new York, accessed : www.bookfi.org , on: 31-03-2016. Hamp-Lyons, L. (1989). Second language writing: Assessment issues. In Kroll, B.

(Ed.), Second language writing Cambridge: Cambridge University Press. Hoskins, S., & Hooff, J. (2005). Motivation and ability: Which students use online learning and what influence does it have on their achievement? British Journal of Educational Technology, 36 (2), 177-192

John W Creswell, Research Design : Qualitative, Quantitative and mixed method approaches, third edition (University of nebrasca liconln data 2009) C.9 Llloyd-Jones, R. 1977. Primary Trait Scoring. In Cooper, Charles R and Lee Odel Meidasari, Venny Eka. Teaching English through New Digital Media (University of

Indraprasta PGRI and a postgraduate student in State Universit y of Jakarta). Miyazoe, T., & Chiyodaku, N., & Anderson, T. (2010). Empirical research on

learners’ perceptions: Interaction equivalency theorem in blended learning, Retrieved June 7,2010, from http://www.eurodl.org/?article=397

Moskal, B.M. (2000). "Scoring rubrics: What, when and how?" Practical Assessment,

Research & Evaluation, 7 (3). Accessed on

http://pareonline.net/getvn.asp?v=7&n=3

Mukminatien, Nur. 2000. The Advantages of Using an Analytic Scoring Procedure in Speaking Assessment, TEFLIN Journal, Vol. XI, No. 1 Agustus 2000. Universitas Negeri Malang.

Nor, Faizah binti Modh and Sumintha Malar Vasu.Teachers’ Preception Of lesson

UsingComputer Assisted Language Learning. Malaysia.University Technology of Malaysia.

Oller J.W., Jr. 1979. LanguageTest at School. London: Longman Group Ltd.

Pandiya, 2013. Rubrics on Scoring English Tests for Four Language Skills. Jurnal Pengembangan Humaniora Vol. 13 No. 1, April. Polines.Press.

Rahman Abdur, Undergraduate Program: “the implementation of reading aloud to

vary the pronunciation practice for students of senior high school in

Mamba`us sholihin muslim boarding school gresik”, Surabaya: state Islamic university sunan ampel surabaya, 2011

(6)

Sa`diyah Istiqomatus, Undergraduate Program : “an analysis of student`s writing

ability in forming irregular verb at the tenth grade of MA Al-Iman Ponorogo”, Surabaya: state Islamic university sunan ampel surabaya, 2012.

Sugiono, Metode Penelitian Pendidikan (Bandung:Alfabeta 2010)

Swales, John. (1992). Language for specific purposes. In W. Bright (Ed.), International encyclopedia of linguistics (Vol. 2, p. 300). New York, Oxford: Oxford University Press.

Tse, D. K., & Wilton, P. C. (1988). Models of consumer satisfaction: An extension, Journal of Marketing Research, 25, 204-212

Underhiil, N. 1987. Testing Spoken Language. cambririge: cambridge university Yang, Y., & Cornelius, L. F. (2004). Students' perceptions towards the quality of

online education: A qualitative approach, Association for Educational Communications and Technology, 27, 861–877.

Young, A., & Norgard, C. (2006). Assessing the quality of online courses from the students' perspective. Internet and Higher Education, 9(2), 107-115

ANALYTIC SCORING PROCEDURE IN SPEAKING ASSESSMENT AT HOSPITALITY PROGRAM OF STATE VOCATIONAL HIGH SCHOOL 1 BUDURAN SIDOARJO.