t bing 1009594 table of content
TABLE OF CONTENTS
Page
APPROVAL OF EXAMINERS ………………………………………...
ii
APPROVAL OF SUPERVISORS ……………………………………...
iii
DECLARATION ………………………………………………………...
iv
ACKNOWLEDGEMENTS ……………………………………………..
v
ABSTRACT ……………………………………………………………...
vi
TABLE OF CONTENTS ………………………………………………..
vii
LIST OF TABLES ……………………………………………………….
xi
LIST OF FIGURES …………………...…………………………………
xiii
CHAPTER I. INTRODUCTION
1
A. Background ……………………………………………………………
1
B. Purpose of the Study ……………………………………………..……
5
C. Research Questions ……………………………………………………
7
D. Scope of Evaluation and Limitations ………………………………….
7
E. Significance of the Study …………………………………..………….
9
F. Clarifications of the Main Terms …………………….……………….
10
G. Evaluation Framework ………………………………………………..
14
CHAPTER II . REVIEW OF THE LITERATURE
15
A. Distance Education…………………………………………………….
15
1. Open and Distance Learning Systems …………………………….
15
2. Distance English Education Program and Its Problems …………..
16
3. Quality Assurance in Distance Education ………………………...
19
B. Program and Curriculum ………………………………………….…..
20
1. Definitions and Components of a Program and a Curriculum …..
20
2. Program/Curriculum Orientations ………………………………...
22
3. Curriculum Development Models ………………………………...
23
4. Curriculum Implementation ……………………………………….
27
Benny Nugraha, 2012
Distance English Education Program Evaluation
At Universitas Terbuka
Universitas Pendidikan Indonesia | Repository.Upi.Edu
Page
C. Program Evaluation …………………………………………………...
28
1. Definitions of Program Evaluation ……………….……………….
28
2. Approaches to Program Evaluation ……………………………….
30
3. Purposes of Program Evaluation …………………………………..
34
4. Functions of Program Evaluation …………………………………
35
5. Evaluation Model Used in the Study ……………………..……….
36
D. Review of Previous Program Evaluations …………………………….
38
E. Synthesized Summary of Theoretical Framework ……………………
45
CHAPTER III. RESEARCH METHOD
49
A. Research Design ………………………………………………………
50
B. Data Collection Procedure …………………………………………….
51
1. Research Site ………………………………………………………
51
2. Population and Sample …………...………………………..……...
51
3. Instrumentation…………………………………………..………...
52
a. Document Survey Form ………………………………………..
53
b. Standard Document Checklist ………………………………….
54
c. Student Survey Questionnaire ………………………………….
56
d. Administrator Survey Questionnaire …………………………..
64
4. Validity ……………………………………………………………
73
5. Reliability …………………………………………………………
73
C. Data Analysis Procedure ……………………………………………...
74
CHAPTER IV. RESULTS AND DISCUSSIONS
75
A. Summary of the Sampling and Instrumentation ………………………
76
1. Sampling …………………………………………………………..
76
2. Survey Administration …………………………………………….
76
3. Response Rate ……………………………………………………..
77
4. Discussion on the Sampling and Instrumentation …………………
78
B. Validity and Reliability ……………………………………………….
81
Benny Nugraha, 2012
Distance English Education Program Evaluation
At Universitas Terbuka
Universitas Pendidikan Indonesia | Repository.Upi.Edu
Page
1. Validity ……………………………………………………………
81
2. Reliability …………………………………………………………
86
3. Discussion on the Validity and Reliability ………………………..
88
C. Demography …………………………………………………………..
89
1. Students’ Demography ……………………………………………
89
2. Administrators’ Demography ……………………………………..
91
3. Discussion on the Demography …………………………………...
92
D. Evaluation Results …………………………………………...………..
93
1. Results of Program Definition Evaluation ………………………..
94
a. Document Survey Results……………………………………..
95
1) Student …………………………………………………….
95
2) Staff ……………………………………….………………
96
3) Objectives …………………………………………………
98
4) Content ……………………………………………………
99
5) Delivery Method ………………………………………….
101
6) Learning Assessment ……………………………………...
103
7) Support System ……………………………………………
104
8) Facilities …………………………………………………...
105
b. Discussion on EEP Design Performance ……………………...
106
2. Results of Program Installation Evaluation ……………………….
108
a. Students’ Perceptions on Input Components ………………..
108
………………………………...…...
b. Administrators’
Perceptions on Input Components …………
111
c. Discussion on EEP Input Component Performance ………….
115
3. Results of Program Process Evaluation …………………………...
117
a. Students’ Perceptions on Process Components …………...…..
118
………………………………...…...
b. Administrators’
Perceptions on Process Components ……..…
120
c. Discussion on EEP Process Component Performances ……….
123
4. Results of Program Product Evaluation …………………………..
124
a. Students’ Perceptions on Product Components ...……………..
124
………………………………...…...
Benny Nugraha, 2012
Distance English Education Program Evaluation
At Universitas Terbuka
Universitas Pendidikan Indonesia | Repository.Upi.Edu
Page
b. Administrators’ Perceptions on Product Components ...………
126
c. Discussion on EEP Product Component Performances …...….
127
E. Additional Evaluation Results ….…………………………...………..
129
1. Content Analysis of Open-Ended Responses …………………….
129
2. Friedman’s Multiple Comparisons for Priority Leveling ………..
131
CHAPTER V. CONCLUSIONS AND RECOMMENDATIONS
141
A. Conclusions …………………………………………………………...
141
B. Recommendations ….…………………………………………………
146
BIBLIOGRAPHY
140
APPENDICES
148
A. Letters …………………………………………………………............
149
B. Expert Judgments …….………….……………………………………
159
CURRICULUM VITAE
161
Benny Nugraha, 2012
Distance English Education Program Evaluation
At Universitas Terbuka
Universitas Pendidikan Indonesia | Repository.Upi.Edu
Page
APPROVAL OF EXAMINERS ………………………………………...
ii
APPROVAL OF SUPERVISORS ……………………………………...
iii
DECLARATION ………………………………………………………...
iv
ACKNOWLEDGEMENTS ……………………………………………..
v
ABSTRACT ……………………………………………………………...
vi
TABLE OF CONTENTS ………………………………………………..
vii
LIST OF TABLES ……………………………………………………….
xi
LIST OF FIGURES …………………...…………………………………
xiii
CHAPTER I. INTRODUCTION
1
A. Background ……………………………………………………………
1
B. Purpose of the Study ……………………………………………..……
5
C. Research Questions ……………………………………………………
7
D. Scope of Evaluation and Limitations ………………………………….
7
E. Significance of the Study …………………………………..………….
9
F. Clarifications of the Main Terms …………………….……………….
10
G. Evaluation Framework ………………………………………………..
14
CHAPTER II . REVIEW OF THE LITERATURE
15
A. Distance Education…………………………………………………….
15
1. Open and Distance Learning Systems …………………………….
15
2. Distance English Education Program and Its Problems …………..
16
3. Quality Assurance in Distance Education ………………………...
19
B. Program and Curriculum ………………………………………….…..
20
1. Definitions and Components of a Program and a Curriculum …..
20
2. Program/Curriculum Orientations ………………………………...
22
3. Curriculum Development Models ………………………………...
23
4. Curriculum Implementation ……………………………………….
27
Benny Nugraha, 2012
Distance English Education Program Evaluation
At Universitas Terbuka
Universitas Pendidikan Indonesia | Repository.Upi.Edu
Page
C. Program Evaluation …………………………………………………...
28
1. Definitions of Program Evaluation ……………….……………….
28
2. Approaches to Program Evaluation ……………………………….
30
3. Purposes of Program Evaluation …………………………………..
34
4. Functions of Program Evaluation …………………………………
35
5. Evaluation Model Used in the Study ……………………..……….
36
D. Review of Previous Program Evaluations …………………………….
38
E. Synthesized Summary of Theoretical Framework ……………………
45
CHAPTER III. RESEARCH METHOD
49
A. Research Design ………………………………………………………
50
B. Data Collection Procedure …………………………………………….
51
1. Research Site ………………………………………………………
51
2. Population and Sample …………...………………………..……...
51
3. Instrumentation…………………………………………..………...
52
a. Document Survey Form ………………………………………..
53
b. Standard Document Checklist ………………………………….
54
c. Student Survey Questionnaire ………………………………….
56
d. Administrator Survey Questionnaire …………………………..
64
4. Validity ……………………………………………………………
73
5. Reliability …………………………………………………………
73
C. Data Analysis Procedure ……………………………………………...
74
CHAPTER IV. RESULTS AND DISCUSSIONS
75
A. Summary of the Sampling and Instrumentation ………………………
76
1. Sampling …………………………………………………………..
76
2. Survey Administration …………………………………………….
76
3. Response Rate ……………………………………………………..
77
4. Discussion on the Sampling and Instrumentation …………………
78
B. Validity and Reliability ……………………………………………….
81
Benny Nugraha, 2012
Distance English Education Program Evaluation
At Universitas Terbuka
Universitas Pendidikan Indonesia | Repository.Upi.Edu
Page
1. Validity ……………………………………………………………
81
2. Reliability …………………………………………………………
86
3. Discussion on the Validity and Reliability ………………………..
88
C. Demography …………………………………………………………..
89
1. Students’ Demography ……………………………………………
89
2. Administrators’ Demography ……………………………………..
91
3. Discussion on the Demography …………………………………...
92
D. Evaluation Results …………………………………………...………..
93
1. Results of Program Definition Evaluation ………………………..
94
a. Document Survey Results……………………………………..
95
1) Student …………………………………………………….
95
2) Staff ……………………………………….………………
96
3) Objectives …………………………………………………
98
4) Content ……………………………………………………
99
5) Delivery Method ………………………………………….
101
6) Learning Assessment ……………………………………...
103
7) Support System ……………………………………………
104
8) Facilities …………………………………………………...
105
b. Discussion on EEP Design Performance ……………………...
106
2. Results of Program Installation Evaluation ……………………….
108
a. Students’ Perceptions on Input Components ………………..
108
………………………………...…...
b. Administrators’
Perceptions on Input Components …………
111
c. Discussion on EEP Input Component Performance ………….
115
3. Results of Program Process Evaluation …………………………...
117
a. Students’ Perceptions on Process Components …………...…..
118
………………………………...…...
b. Administrators’
Perceptions on Process Components ……..…
120
c. Discussion on EEP Process Component Performances ……….
123
4. Results of Program Product Evaluation …………………………..
124
a. Students’ Perceptions on Product Components ...……………..
124
………………………………...…...
Benny Nugraha, 2012
Distance English Education Program Evaluation
At Universitas Terbuka
Universitas Pendidikan Indonesia | Repository.Upi.Edu
Page
b. Administrators’ Perceptions on Product Components ...………
126
c. Discussion on EEP Product Component Performances …...….
127
E. Additional Evaluation Results ….…………………………...………..
129
1. Content Analysis of Open-Ended Responses …………………….
129
2. Friedman’s Multiple Comparisons for Priority Leveling ………..
131
CHAPTER V. CONCLUSIONS AND RECOMMENDATIONS
141
A. Conclusions …………………………………………………………...
141
B. Recommendations ….…………………………………………………
146
BIBLIOGRAPHY
140
APPENDICES
148
A. Letters …………………………………………………………............
149
B. Expert Judgments …….………….……………………………………
159
CURRICULUM VITAE
161
Benny Nugraha, 2012
Distance English Education Program Evaluation
At Universitas Terbuka
Universitas Pendidikan Indonesia | Repository.Upi.Edu