Manajemen | Fakultas Ekonomi Universitas Maritim Raja Ali Haji joeb.84.6.323-331
Journal of Education for Business
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Effectiveness of Web-Based Courses on Technical
Learning
Monica Lam
To cite this article: Monica Lam (2009) Effectiveness of Web-Based Courses on Technical
Learning, Journal of Education for Business, 84:6, 323-331, DOI: 10.3200/JOEB.84.6.323-331
To link to this article: http://dx.doi.org/10.3200/JOEB.84.6.323-331
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 40
View related articles
Citing articles: 6 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 11 January 2016, At: 22:56
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
EffectivenessofWeb-BasedCourseson
TechnicalLearning
MONICALAM
CALIFORNIASTATEUNIVERSITY
SACRAMENTO,CALIFORNIA
ABSTRACT.Theauthorinvestigated
theeffectivenessofWeb-basedcourseson
technicallearning.Theregressionresults
showthatthedeliveryformat(Web-based
ortraditionalclassroomcourses)hasno
significanteffectonstudentperformance.
However,althoughgenderisasignificant
predictorintraditionalclassroomcourses,
itseffectdisappearsinWeb-basedcourses.
ThereisevidencethatWeb-basedcourses
canbeconducivetotheleaningprocessof
technicalknowledgeforfemalestudents.
Forthehigh-GPAsubgroup,thepredictors
ofethnicity,GPA,andproblem-solving
questionsasanevaluationmethodwere
positivelyassociatedwithperformance.
Keywords:ethnicity,evaluationmethod,
genderdifference,studentperformance,
Web-basedlearning
Copyright©2009HeldrefPublications
T
he Web influences every aspect
of life, including how individuals learn. At present, individuals can
earn a full degree by way of the Web.
TraditionaluniversitiesalsoofferWebbased (WB) courses to enhance their
deliverychannels.Inthepresentstudy,
I investigated the effectiveness of WB
courses on students’ technical learning
as measured by students’ final examination scores. Along with the delivery method factor (WB or traditional
classroom[TC]courses),Ialsoadopted
students’cumulativeGPA(foracademic
standing),gender,ethnicity,andevaluation method (multiple-choice or problem-solving questions) as the predictor
variables. I applied multiple regression
analysestotheentiredatasetandsubsetsofdata.
LiteratureReview
The WB learning phenomenon has
become increasingly prevalent and significant. Many indicators, including the
percentage of colleges that offer WB
learning, expenditure on WB learning
technology,WB course enrollment, and
online tuition and fees earned by educational institutes, show the dramatic
upward trends of WB learning and its
variants (Quinn et al., 2006; Symonds,
2001).WB learning has also penetrated
traditional brick-and-wall campuses,
which are proud of their classroom
teaching. In the United States, the University of Maryland, the largest state
university, offers students more than 70
different degree and certificate options
bywayoftheWeb.Professionaldegrees,
which rely on discussion, interaction,
networking,andcasestudiesasprimary
learning techniques, are no exception.
Forexample,ConcordLawSchooloffers
onlinelawdegrees.DukeUniversityhas
a global executive MBA program that
allowsworkingexecutivestofinish65%
ofthecurriculumovertheWeb(Arbaugh,
2000; McCallister & Matthews, 2001;
Symonds).Manymoreequivalentexamplescanalsobefound.
WB learning has its advantages and
disadvantages.Onthepositiveside,WB
learning has no classroom restrictions.
Studentscanlearnattheirownpaceand
at a convenient time and place.This is
especially important for working individualsandnontraditionalstudentswho
arephysicallyseparatedfromcampuses
or cannot frequently commute to campuses.WBlearningalsohasthebenefit
of transferring the control to students
(Kochtanek&Hein,2000;Lin&Hsieh,
2001).Studentscanmovebackandforth
between Web pages, spend as much
timeasnecessaryonacertaintopic,and
revisit pages for difficult topics. WB
courses also allow instructors to organize the course content into a logical
and written format that is beneficial to
studentswhodonothavegoodlistening
July/August2009
323
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
skills.Alternatively,WBcoursesrequire
studentstohavegoodreadingskillsand
self-discipline. Students may also have
isolation problems and technical difficulties(Palloff&Pratt,1999;Sweeney
& Ingram, 2001). The basic question
in the present study is whether WB
courses are better than TC courses in
termsofdeliveringtechnicalknowledge
and,ifso,whatkindsofstudentsbenefit
the most. In the remainder of this section,Ireviewrelevantresearchfindings
intheliteraturefortheaforementioned
researchquestion.Toprovideafocused
review,Iuseadiscussionframeworkof
fourfactorstodescriberesearchstudies.
Thefourfactorsinthediscussionframework include subject matter, measures
forcourseeffectiveness,studentcharacteristics, and research results. I present
a brief explanation of the discussion
framework before the literature review.
Thereviewrevealswhichaspectsofthe
research question in the present study
remainunanswered.
Thefirstfactorinthediscussionframework is subject matter. In their metaanalysis of WB instruction, Sitzmann,
Kraiger, Stewart, and Wisher (2006)
differentiated between declarative and
proceduralknowledgetoinvestigatethe
effectivenessofWBinstruction.Declarativeknowledgereferstothememoryof
facts,principles,andrelationsofknowledge elements and cognitive strategies
for accessing and applying knowledge.
Alternatively, procedural knowledge
referstohowtoperformatask,including compilation steps, traversal strategies, and optimization methods. In the
presentstudy,myaimwastounderstand
the effect of WB courses on technical
learning, which falls into the domain
of procedural knowledge. The second
factorofeffectivenessinthediscussion
framework can be classified into the
twocategoriesofperformanceandperception.Performanceisstudents’actual
learningresultsfromobservablebehavior,suchasexaminationscores.Perception, on the contrary, is opinion-based
andsubjectiveconcerningstudents’satisfaction and perceived usefulness of
WBcourses.Imeasuredstudents’actual
performancefromacomprehensivefinal
examinationinaprogrammingclass.For
studentcharacteristics,thethirdfactorin
the discussion framework, there were
324
JournalofEducationforBusiness
variablessuchaslearningstyle,ethnicity, gender, age, prior knowledge, and
learningskills.Theresearchrationalefor
investigatingstudentcharacteristicswas
based on the assumption that students
with different profiles would respond
differently to WB courses, leading to
different degrees of performance and
perception. Regarding research results
as the fourth factor in the discussion
framework,theoutcomesareasfollows:
WBcoursesaremoreeffectivethanTC
courses, TC courses are more effective
than WB courses, or TC courses have
thesameeffectivenessasWBcourses.
NoPerformanceDifferenceBetween
WBandTCStudents
Six recent studies (Friday, FridayStroud, Green, & Hill, 2006; Jones,
Moeeni,&Ruby,2005;Piccoli,Ahmad,
& Ives, 2001; Priluck, 2004; Scheines,
Leinhardt,Smith,&Cho,2005;Smeaton&Keogh,1999)reportedthatthere
was no student performance difference
betweenWBandTCcourses.Smeaton
and Keogh, Piccoli et al., and Jones et
al. had IT-related courses as the subject matter, whereas Priluck, Scheines
etal.,andFridayetal.useddeclarative
knowledge as the subject matter. Priluckalsoinvestigatedstudents’satisfaction,whichwashigherforTCthanfor
WBcoursesintermsofteambuilding,
criticalthinking,oralandwrittencommunications, global perspective, and
social interaction. That provides some
evidencethathighsatisfactiondoesnot
necessarily lead to high performance
inWB orTC courses. Lee (2001) also
foundthatstudents’satisfactionhadno
relation to self-reported achievement
in WB courses, which reinforces the
claim that performance, self-reported
or actual, is not necessarily related to
satisfactionwithWBcourses.
WBLearningBetterThanTCLearning
There are studies showing that studentsinWBcoursesachievebetterperformance than in TC courses (Bryan,
Campbell, & Kerr, 2003; Chou & Liu,
2005; Lockyer, Patterson, & Harper,
2001).Lockyeretal.studiedtheeffect
ofWBlearninginundergraduatehealth
care courses and concluded that WB
studentsscoredhigherthandidTCstu-
dents. Regarding student participation,
Lockyer et al. found that WB students
generated higher quality discussion by
providing more in-depth content and
references,whereasTCstudentsgenerated a higher quantity of discussion.
For Bryant et al., the subject matter
was an introductory information system course for undergraduate students.
That study concluded that WB learning was significantly better than TC
learning for concept tests, TC learning
was marginally better than WB learningforgroupproject,andWBlearning
wasjustaseffectiveasTClearningfor
activity folio. Chou and Liu measured
differences in student performance and
perception between WB learning and
TClearningforahighschoolIT-related
course.ChouandLiureportedthatWB
learningwasbetterthanTClearningfor
allvariablesforlearningeffectiveness.
TCLearningBetterThanWBLearning
Therearenotmanystudiesreporting
better performance from TC learning
compared with WB learning. Sweeney
and Ingram (2001) investigated marketing students’ perceived learning
effectiveness in WB and TC learning.
Sweeney and Ingram found that TC
learning was perceived to have higher
learning effectiveness in a tutorial setting.Bryanetal.(2003)foundthatTC
learningwasmarginallybetterthanWB
learning for group projects. Maki and
Maki (2002) compared the test scores
betweenWB andTC learning in introductorypsychologycourses,asmoderatedbystudents’comprehensionskills.
Maki and Maki determined that studentswithhighercomprehensionskills
performed better in WB than in TC
courses.However,comprehensionskills
were not significantly associated with
satisfaction,andallstudents,regardless
oftheircomprehensionskills,preferred
TCtoWBcourses.
EffectsofStudentCharacteristics
I report recent research findings of
the effects of student characteristics on
performance and perception for WB
and TC learning. Wang and Newlin’s
(2000)studyaboutpsychologystudents
revealed no student demographic differencesbetweenWBandTClearning.
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
Bryan et al.’s (2003) study determined
that academic standing (measured by
tertiary entrance score) significantly
affected performance. Alternatively,
Jonesetal.(2005)foundthatacademic
standing (as measured by GPA), age,
workhours,andpreviouscourseshadno
effectonstudents’performanceforWB
and TC learning in an undergraduate
telecommunication course. Jones et al.
didnotconsidergenderandethnicityas
predictors. As suggested by theoretical
analyses (Prinsen, Volman, & Terwel,
2007)andexperimentalresearchresults
(Arbaugh 2000; Lu, Yu, & Liu, 2003;
Wallace & Clariana, 2005; Wernet,
Olliges, & Delicath, 2000), gender and
ethnicity may have significant effects
onperformancedifferencebetweenWB
andTClearninginregardtoprocedural
knowledge, which deserves research
attention. In a study comparing performance difference between a paper test
andanonlinetestforanundergraduate
informationsystemscourse(Wallace&
Clariana), researchers discovered that
non-White female participants scored
the lowest for online tests on the midterm,butthehighestforonlinetestsfor
thefinal.However,WallaceandClariana
onlyinvestigatedthedifferencebetween
paper and online tests, not the difference between WB and TC learning. In
other words, the treatment difference
was not about WB and TC instruction,
butjustaboutthemediumfortestdelivery. When the investigation target was
WB learning only (i.e., no comparison
between WB and TC learning), Lu et
al. concluded that learning style, gender, age, job status, year of admission,
onlineexperience,andMISpreparation
hadnoeffectonstudentperformancein
an MBA or MIS course. In Lu et al.’s
study,theonlystudentcharacteristicthat
had a significant effect was ethnicity,
whichidentifiedWhitesashavingbetter
performance than Blacks inWB courses.InanotherWB-onlystudy,Arbaugh
showed that older female students may
havestrongerleaningexperiencesinWB
courses. Nevertheless, Arbaugh only
covered students’ perception, not their
actualperformance.Wernetetal.studied
socialworkstudents’satisfactioninWB
andTCcoursesandconcludedthatolder
nontraditionalstudentsfoundWBlearningmoreappealingandhadhighersat
isfaction with it.A meta-analysis study
regarding gender differences on computer-supported collaborative learning
concluded that WB courses provided a
more equitable discussion environment
in which women felt more comfortable
participating(Prinsenetal.).
Insummary,themajorityofresearch
results from existing literature, which
indicates thatWB learning is as effective as or equivalent to TC learning
forstudentperformance.However,itis
not clear for most studies whether the
subjectmatterisdeclarativeknowledge,
procedural knowledge, or a mixture of
both.WhenTClearningwasconsidered
as more effective than WB learning,
it was for perceived learning, group
project,andsatisfaction.Regardingstudent characteristics, some studies have
shownthatage(Jonesetal.,2005;Luet
al.,2003),learningstyle(Luetal.),and
priorknowledgeandskills(Jonesetal.;
Luetal.)arenotsignificantlyassociated
withperformanceinWBorTCcourses,
althoughagemaybeafactorforsatisfaction (Arbaugh, 2000; Wernet et al.,
2000).Forothercharacteristics,suchas
academicstanding,gender,andethnicity,therearemixedoruncertainresults.
Foracademicstandingasaperformance
predictor,Bryanetal.(2003)reportedit
tobesignificant,butJonesetal.reported it to be insignificant. Similarly, for
ethnicity as a performance predictor,
Lu et al. reported it to be significant,
but Wang and Newlin (2000) reported
it to be insignificant. As for gender
as a performance predictor, studies for
gendereffecthavefocusedonWebstudentsonly(Luetal.),testmediumonly
(Wallace&Clariana,2005),ordeclarative knowledge only (Scheines et al.,
2005). The effect of gender on performancedifferencebetweenWBandTC
learning for technical subjects has not
beendetermined.
StatementoftheProblem
In the present research, I investigated the effect of student characteristics
(GPA,ethnicity,gender,hitrate,andread
rate), delivery modes (WB orTC), and
evaluation methods (problem-solving
or multiple-choice questions) for the
overall performance of students (final
exam score) in technical undergradu-
atecourses.Thepredictorvariablesand
dependentvariablearefullydescribedin
theMethodsection.Thesignificanceof
this research is threefold. First, educatorsneedtounderstandtheeffectiveness
of WB learning for different students
in different subjects. The present study
sheds light on the effectiveness of WB
learning on technical subjects such as
programming and information system
development.Technical learning, by its
nature, fits the definition of procedural
knowledge. There are many aspects of
technical learning that can be difficult
to deliver by way of Web pages. For
example, debugging, as an important
skillforstudentstomasterinprogramming classes, may be more effectively taught in a face-to-face laboratory
environment. Alternatively, the logical
understandingofprogrammingfallsinto
the area of higher order skills, which
is claimed to be an effective learning
objective for WB courses (Kao, 2002).
Inthepresentstudy,Iaimedtofindthe
overalleffectivenessofWBlearningfor
technicalsubjects,whichhavebothpros
and cons for WB delivery. Second, the
presentstudyprovidesguidanceforthe
pedagogy of WB courses for technical subjects. Programming courses are
usually difficult for students to master. Understanding how different question formats can promote learning is
beneficialtoinstructors’teachingplans.
The treatment of evaluation method in
the present study provides information
about this issue. The third contribution
ofthepresentresearchisunderstanding
theeffectofGPA,gender,andethnicity
on performance difference for procedural knowledge between WB and TC
learning,whichhasnotbeenconfirmed
intheliterature.
METHOD
The present research is an empirical
study using data from an undergraduate elective programming course that I
taughtfor2yearsandapplicationdevelopment using Visual Basic. I collected
data from class records and students’
recordsintheuniversitysystem.Thepredictor variables included students’ gender, ethnicity, GPA, hit rate, read rate,
delivery mode, and evaluation method.
The dependent variable was students’
July/August2009
325
final examination scores at the end of
a semester. I used analyses of variance
(ANOVAs) and regression analyses as
thestatisticaltoolsforthisresearch.
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
Participants
Participants were 364 students from
sixWBandthreeTCcoursesin2academic years (221 WB students, 143
TC students). Tables 1–3 provide the
descriptivestatisticsforparticipants,by
ethnicity, gender, delivery method, and
evaluation method. The course was an
undergraduate elective programming
classforMISmajorsusingASP.Netto
develop Web applications. I designed
andtaughtalltheWBandTCsections.
The prerequisite for enrolling in the
targeted class was a passing grade in
an advanced Visual Basic class. The
only requirement for enrolling in WB
courses was Internet access. Students
chose freely whether to enroll in WB
orTCcourses.Duringthestudyperiod,
there were no cases in which students
couldnotenrollinaWBorTCcourse
becausethesectionwasfull.
Treatment
I designed the WB course using
WebCT. The main page of the WB
coursehastheiconsofsyllabus,assignments, PowerPoint slides, exams and
quizzes,discussionboard,e-mail,sampleprograms,labexercises,andacalendar.TheWBcoursedividedintoseven
sections. Students were advised to finish each section following a flowchart
thatoutlinedstepbystepwhattodoby
when. PowerPoint slides were annotatedbuthadnosound.Eachsectionhada
follow-upquizthatstudentshadtotake
byacertaindeadline.Follow-upquizzes
wereopen-bookandopen-note.Correct
answerstoquizzeswereavailablefrom
WebCTafterthedeadline.Becausequiz
questions were randomized from a test
bank,thechanceoftwostudentsgetting
the same questions in a quiz was slim.
Quiz records show that approximately
95%ofstudentsfinishedtheirquizzeson
time. Students accessed quiz questions
and their answers after the deadline,
eventhoughtheydidnottakethequiz.
Laboratory exercises were designed
for students to practice programming
concepts and techniques. WB students
326
JournalofEducationforBusiness
were advised to do their lab exercises
in one of the student labs any time by
a certain deadline.Although there was
no way for me to check whether students finished their laboratory exercises, that follow-up quiz questions were
partially based on laboratory exercises
providedincentiveforstudentstowork
on laboratory exercises on their own.
WB students were encouraged to email the instructor whenever they had
questionsorproblemswiththeWebCT
system.Somestudentscomplainedthat
theWebCT system did not record quiz
scores correctly or the screen froze up
after a few questions. I allowed WB
students to delete approximately 15%
ofthelowestquizscorestocompensate
for any technical problems that were
out of their control.WB students were
requiredtocomebacktotheclassroom
totakeallexaminationsinperson.
TheTCsectionofthecoursehadall
thesameteachingmaterialsastheWB
section except for a few differences.
First,IpresentedallPowerPointslides
intheclassroom,andsecond,Iadministered all quizzes in the classroom.
Quiz questions were also randomized
in the lecture section. Third, students
didtheirexercisesinalaboratoryenvironmentthatImonitored.Iwasavailabletoanswerstudentquestionswhen
theywereworkingontheirexercisesin
thelaboratory.
HypothesesforGPA,Ethnicity,
Gender,HitRate,andReadRate
This section describes GPA, ethnicity, gender, hit rate, and read rate
as the predictor variables for student
performance. The ethnicity predictor
has Black, White, Hispanic, Indian or
MiddleEastern,andAsianasvariables
in the present study. I observed that
in technical subjects, students with the
sameethnicitytendedtocollaborateand
assistoneanother.Thefactorofundergraduate classes is likely to intensify
the phenomenon of mutual assistance
because undergraduate students have
more opportunities to stay on campus
thandograduatestudents.BecauseWB
learningisnotyetcommononthecampus on which this experiment was carried out, undergraduate students who
enrolledinWBcoursesstillhadplenty
ofopportunitiestomeetoneanother.In
technicalWBcourses,inwhichstudents
havetorelyonthemselvestobeactive
learners,studentswiththesameethnicitymaydeveloptheirlearningnetwork
forcollaboration.Consideringallofthe
aforementioned issues, I hypothesized
TABLE1.DescriptiveStatistics
forCategoricalVariables(N=
364)
Variable
n
Ethnicity
Black
White
Hispanic
IndianorMiddleEastern
Asian
Gender
Male
Female
Deliverymethod
Web-based
Traditionalclassroom
Evaluationmethod
Problemsolving
Multiplechoice
12
133
40
15
164
209
155
221
143
150
214
TABLE2.Students,byDelivery
andEvaluationMethods
Delivery
method
Web-based
Traditional
classroom
Evaluationmethod
Problem
solving
Multiple
choice
110
114
40
103
TABLE3.EthnicityandGender
DistributionforWeb-based
(WB)andTraditionalClassroom(TC)Students
Variable
TC(%)
Ethnicity
Black
2.79
White
31.46
Hispanic
10.48
IndianorMiddle
Eastern
4.89
Asian
50.34
Gender
Male
56.64
Female
43.36
WB(%)
3.61
39.81
11.31
3.61
41.62
57.91
42.08
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
that ethnicity would have a significant
effect on undergraduate technical WB
learning. Regarding gender as a predictor for performance, the literature
has not provided a complete picture.
I hypothesized there would be a significant gender effect on performance
betweenWBandTCstudents.
GPAisstudents’cumulativeGPAup
tothesemestertheytooktheprogramming course under investigation. GPA
measures students’ overall academic
standing on the basis of all courses
that they have taken at the university.
Overall academic standing indicates a
student’s discipline, study habit, and
performance,whichusuallyisareliable
predictor for new academic endeavor.
HitandreadratesarevariablesforWB
students only. Hit rate is the number
ofaWBstudent’slog-onsessions,and
read rate is the number of teaching
materials a WB student retrieves to
readfromtheWBcourse.Hitandread
rates measure a student’s enthusiasm
to access course materials on theWeb.
I hypothesized that GPA, hit rate, and
readratewouldbesignificantpredictors
forperformanceinWBcourses.
HypothesisforDeliveryMode
andEvaluationMethod
The delivery mode of WB or TC
learning is a major treatment in the
present study. I hypothesized that different delivery modes would generate
differentperformance.Thepredictorof
evaluationmethod,asanothertreatment
in the present study, determines the
question format in the midterm: multiple-choice or problem-solving questions. Midterms prepare students for
theirfinalcomprehensiveexamination.
Because problem-solving questions
involve in-depth analysis of problems
and practice on solution development,
I hypothesized that problem-solving
questions would be associated with
higheroverallperformancethanwould
multiple-choicequestions.
OverallPerformanceDifference
The total score in the final comprehensive examination was used as the
dependentvariableinthepresentstudy.
The final comprehensive examination
was the same for all students in all
semesters. The final comprehensive
examination had a variety of question
formats including multiple choice,
problem solving, programming, and
concept definitions. I chose to use the
totalscoreratherthanthefinalgradeas
thedependentvariablebecausethetotal
scorewasmorecompatibleamongdifferentsectionsthanwasthefinalgrade,
which may have to be normalized in
somesections.
AnalyticalProcedure
Thefirststepinthedataanalysisprocess was to check student distribution
intermsofgender,ethnicity,GPA,and
total score betweenWB andTC learning. I applied anANOVA to determine
whether there was a significant GPA
andtotalscoredifferencebetweenWB
and TC learning. In the second step, I
applied multiple regression analyses to
theentiredatasetwithallvariables.In
thethirdstep,IappliedmultipleregressionanalysestothedatasubsetsofWB
and TC learning. In the fourth step, I
applied multiple regression analyses to
the student subgroups of low, medium,
and high GPA. All normal probability
plots for regression models showed no
sign of assumption violation and problematicresidues.
1,thepresentstudyhadasmallsample
size for Black (n = 12), Hispanic (n =
40),andIndianorMiddleEastern(n=
15) students. To check for significant
mean differences for GPA and total
score betweenWB and TC students, I
performed anANOVA. Table 4 shows
the maximums, minimums, standard
deviations, and means for GPA and
totalscore,asclassifiedbyWBandTC
learning.Themeandifferencesbetween
WBandTClearningforGPAandscore
are not significant. The insignificant
totalscoredifferencebetweenWBand
TC learning confirms those research
resultsfromtheliteratureclaimingWB
learning as equivalent to TC learning
for learning procedural knowledge.
Table5showsthedescriptivestatistics
forthevariableshitrateandreadratein
WBlearning.
GPAasSignificantPredictorfor
AllStudents,WBStudents,and
TCStudents
Table 6 shows the regression results
forallstudentsinthedataset.Model1.1
inTable6hasallthepredictorvariables.
TABLE4.DescriptiveStatistics
forGPA(Predictor)andScore
(DependentVariable)
RESULTS
Statistics
GPA Totalscore
GeneralDataDistribution
Maximum
Minimum
SD
M
MDifference
WB
TC
p
4.00
1.67
0.52
2.86
102.10
28.33
12.53
71.23
2.85
2.86
.82
71.36
71.01
.79
Tables 1–3 show the student distribution by ethnicity, gender, delivery
method, and evaluation method in the
entire data set. There were 364 studentsintheentiredatasetinwhichthe
majority was Asian (164) and White
(133).Thereweremoremale(n=209)
andWB (n = 221) students than there
werefemale(n=155)andTC(n=143)
students in the entire data set. As for
the evaluation method shown in Table
2, the data set has more students with
multiple-choice questions (n = 217)
than with problem-solving questions
(n = 150) in midterm examinations.
ComparingethnicityandgenderdistributionforWBandTCstudentsinTable
3, the main difference is the higher
percentage of White students and the
lower percentage of Asian students in
WB courses. Also, as shown in Table
Note. WB=Web-based;TC=traditional
classroom.
TABLE5.DescriptiveStatistics
forHitandReadVariablesfor
Web-basedStudents
Statistics
Maximum
Minimum
SD
M
Hit
583.00
33.00
179.34
87.62
Read
62.00
0.00
36.89
16.27
July/August2009
327
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
BecauseGPAisthedominantpredictor,
IremoveditfromModel1.2torevealthe
significanceforothervariables.Models
1.1 and 1.2 are significant, but Model
1.2hadasmalladjustedR2(.037).Only
predictorswithasignificancelevelof≤
.05werelistedinregressionresults.In
Model1.1,whichisthemodelwithall
predictorsforallstudents,onlyGPAisa
significant predictor at the significance
level of zero, and it has a standardized
coefficientof.57.InModel1.2,which
hasallpredictorsexceptGPAforallstudents,Whitehaspositiveeffect.Inother
words,whenGPAisignored,Whitestudentstendtohavehigherscores.
Table 7 shows the regression results
forWBstudents.InModel2.1(withall
predictors),themodelissignificant,F(1)
=0,p=0.Again,GPAistheonlysignificantpredictoratasignificancelevelof
zero.TheresultsfromModel2.1(WB)
aresimilartothosefromModel1.1(all
students) with GPA as the only highly
significantandpositivepredictor.Model
2.2 (without GPA for WB students) is
alsosimilartoModel1.2(withoutGPA
forallstudents),ignoringtheBlackand
IndianorMiddleEasterneffectsbecause
oftheirsmallsamplesizes.
Table 8 shows the regression results
forTCstudents.Model3.1isthemodel
withallpredictorsforTCstudents.The
model is highly significant, F(3) = 0
(differentpredictorshavedifferentpvalues; see Table 8), with a high adjusted
R2(.462).Female,problemsolving,and
GPA are the three significant predictors, which have –.16, .267, and .612
as the standardized coefficients, respectively. The standardized coefficients in
Model 3.1 indicate the following: (a)
Female students in TC courses tend to
havelowscores;(b)problemsolvingas
theevaluationmethodisassociatedwith
highscores;and(c)GPAisstillthebest
predictor for score. Comparing Model
3.1 (all predictors for TC) with Model
2.1 (all predictors for WB students), I
noticedthatthepredictionpowerofGPA
is stronger (β = .612) in TC courses
than inWB (β = .549) courses.Whereas female and problem solving have no
significant effect on WB learning, they
respectively have significantly negative
andpositiveeffectsonTClearning.The
insignificance of female and problem
solvingonWBlearningmaybebecause
328
JournalofEducationforBusiness
oftheflexibilityoflearningpaceinWB
courses, which eliminates the learning difficulties for some students inTC
courses.InModel3.2,femaleandproblem solving are still the significant predictorsforTClearningaftertheremoval
oftheGPApredictor.
PredictorPowerofHigh,Medium,
andLowGPA
Because GPA is a highly significant
predictor, I classified students into
high-, medium-, and low-GPA groups
to perform further analyses: high GPA
≥ 3.1; medium GPA ≥ 2.5, but ≤ 3.1;
and low GPA ≤ 2.5. Table 9 shows
theregressionresultsforthehigh-GPA
(Model4.1),medium-GPA(Model4.2),
and low-GPA (Model 4.3) groups. All
models are highly significant, but only
thehigh-GPAgrouphasadecentadjustedR2(.302).Thesignificantpredictors
inthehigh-GPAgroupareAsian,problem solving, and GPA. All significant
predictors in the high GPA group have
positive standardized coefficients. In
otherwords,inthehigh-GPAgroup,the
predictorsofAsian,GPA,andproblem
solving as the evaluation method have
asignificantimpactonperformance.In
the medium-GPA group, no predictor
is significant. In the low-GPA group,
no predictor has a positive effect, but
TABLE6.RegressionResultsforAllStudents
Model
Model
significance(F)
1.1(withall
variables)
1.2(without
GPA)
df
AdjustedR2
Variable,*p
Standardized
coefficients
0.000
1
.323
GPA,0
0.001
3
.037
Black,.031
White,.008
IndianorMiddle
Eastern,.037
.57
–.113
.140
.109
*
p≤.05
TABLE7.RegressionResultsforWeb-basedStudents
Model
Model
significance(F)
2.1(withall
variables)
2.2(without
GPA)
df
AdjustedR2
Variable,*p
0.000
1
.298
GPA,0
0.004
2
.041
Black,.071
White,.012
Standardized
coefficients
.549
–.121
.17
*
p≤.05
TABLE8.RegressionResultsforTraditionalClassroomStudents
Model
Model
significance(F)
3.1(withall
variables)
3.2(without
GPA)
*
p≤.05
0.000
0.001
df
AdjustedR2
3
2
.462
.086
Variable,*p
Standardized
coefficients
Women,.011
–.160
Problemsolving,0
.267
GPA,0
.612
Women,.05
–.159
Problemsolving,.001 .264
TABLE9.RegressionResultsforAllStudents,ClassifiedbyGPA
Model
Model
significance(F) df
34.1(highGPA;
GPA≥3.1;
n=118)
4.2(medium
GPA;2.5≤
GPA
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Effectiveness of Web-Based Courses on Technical
Learning
Monica Lam
To cite this article: Monica Lam (2009) Effectiveness of Web-Based Courses on Technical
Learning, Journal of Education for Business, 84:6, 323-331, DOI: 10.3200/JOEB.84.6.323-331
To link to this article: http://dx.doi.org/10.3200/JOEB.84.6.323-331
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 40
View related articles
Citing articles: 6 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 11 January 2016, At: 22:56
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
EffectivenessofWeb-BasedCourseson
TechnicalLearning
MONICALAM
CALIFORNIASTATEUNIVERSITY
SACRAMENTO,CALIFORNIA
ABSTRACT.Theauthorinvestigated
theeffectivenessofWeb-basedcourseson
technicallearning.Theregressionresults
showthatthedeliveryformat(Web-based
ortraditionalclassroomcourses)hasno
significanteffectonstudentperformance.
However,althoughgenderisasignificant
predictorintraditionalclassroomcourses,
itseffectdisappearsinWeb-basedcourses.
ThereisevidencethatWeb-basedcourses
canbeconducivetotheleaningprocessof
technicalknowledgeforfemalestudents.
Forthehigh-GPAsubgroup,thepredictors
ofethnicity,GPA,andproblem-solving
questionsasanevaluationmethodwere
positivelyassociatedwithperformance.
Keywords:ethnicity,evaluationmethod,
genderdifference,studentperformance,
Web-basedlearning
Copyright©2009HeldrefPublications
T
he Web influences every aspect
of life, including how individuals learn. At present, individuals can
earn a full degree by way of the Web.
TraditionaluniversitiesalsoofferWebbased (WB) courses to enhance their
deliverychannels.Inthepresentstudy,
I investigated the effectiveness of WB
courses on students’ technical learning
as measured by students’ final examination scores. Along with the delivery method factor (WB or traditional
classroom[TC]courses),Ialsoadopted
students’cumulativeGPA(foracademic
standing),gender,ethnicity,andevaluation method (multiple-choice or problem-solving questions) as the predictor
variables. I applied multiple regression
analysestotheentiredatasetandsubsetsofdata.
LiteratureReview
The WB learning phenomenon has
become increasingly prevalent and significant. Many indicators, including the
percentage of colleges that offer WB
learning, expenditure on WB learning
technology,WB course enrollment, and
online tuition and fees earned by educational institutes, show the dramatic
upward trends of WB learning and its
variants (Quinn et al., 2006; Symonds,
2001).WB learning has also penetrated
traditional brick-and-wall campuses,
which are proud of their classroom
teaching. In the United States, the University of Maryland, the largest state
university, offers students more than 70
different degree and certificate options
bywayoftheWeb.Professionaldegrees,
which rely on discussion, interaction,
networking,andcasestudiesasprimary
learning techniques, are no exception.
Forexample,ConcordLawSchooloffers
onlinelawdegrees.DukeUniversityhas
a global executive MBA program that
allowsworkingexecutivestofinish65%
ofthecurriculumovertheWeb(Arbaugh,
2000; McCallister & Matthews, 2001;
Symonds).Manymoreequivalentexamplescanalsobefound.
WB learning has its advantages and
disadvantages.Onthepositiveside,WB
learning has no classroom restrictions.
Studentscanlearnattheirownpaceand
at a convenient time and place.This is
especially important for working individualsandnontraditionalstudentswho
arephysicallyseparatedfromcampuses
or cannot frequently commute to campuses.WBlearningalsohasthebenefit
of transferring the control to students
(Kochtanek&Hein,2000;Lin&Hsieh,
2001).Studentscanmovebackandforth
between Web pages, spend as much
timeasnecessaryonacertaintopic,and
revisit pages for difficult topics. WB
courses also allow instructors to organize the course content into a logical
and written format that is beneficial to
studentswhodonothavegoodlistening
July/August2009
323
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
skills.Alternatively,WBcoursesrequire
studentstohavegoodreadingskillsand
self-discipline. Students may also have
isolation problems and technical difficulties(Palloff&Pratt,1999;Sweeney
& Ingram, 2001). The basic question
in the present study is whether WB
courses are better than TC courses in
termsofdeliveringtechnicalknowledge
and,ifso,whatkindsofstudentsbenefit
the most. In the remainder of this section,Ireviewrelevantresearchfindings
intheliteraturefortheaforementioned
researchquestion.Toprovideafocused
review,Iuseadiscussionframeworkof
fourfactorstodescriberesearchstudies.
Thefourfactorsinthediscussionframework include subject matter, measures
forcourseeffectiveness,studentcharacteristics, and research results. I present
a brief explanation of the discussion
framework before the literature review.
Thereviewrevealswhichaspectsofthe
research question in the present study
remainunanswered.
Thefirstfactorinthediscussionframework is subject matter. In their metaanalysis of WB instruction, Sitzmann,
Kraiger, Stewart, and Wisher (2006)
differentiated between declarative and
proceduralknowledgetoinvestigatethe
effectivenessofWBinstruction.Declarativeknowledgereferstothememoryof
facts,principles,andrelationsofknowledge elements and cognitive strategies
for accessing and applying knowledge.
Alternatively, procedural knowledge
referstohowtoperformatask,including compilation steps, traversal strategies, and optimization methods. In the
presentstudy,myaimwastounderstand
the effect of WB courses on technical
learning, which falls into the domain
of procedural knowledge. The second
factorofeffectivenessinthediscussion
framework can be classified into the
twocategoriesofperformanceandperception.Performanceisstudents’actual
learningresultsfromobservablebehavior,suchasexaminationscores.Perception, on the contrary, is opinion-based
andsubjectiveconcerningstudents’satisfaction and perceived usefulness of
WBcourses.Imeasuredstudents’actual
performancefromacomprehensivefinal
examinationinaprogrammingclass.For
studentcharacteristics,thethirdfactorin
the discussion framework, there were
324
JournalofEducationforBusiness
variablessuchaslearningstyle,ethnicity, gender, age, prior knowledge, and
learningskills.Theresearchrationalefor
investigatingstudentcharacteristicswas
based on the assumption that students
with different profiles would respond
differently to WB courses, leading to
different degrees of performance and
perception. Regarding research results
as the fourth factor in the discussion
framework,theoutcomesareasfollows:
WBcoursesaremoreeffectivethanTC
courses, TC courses are more effective
than WB courses, or TC courses have
thesameeffectivenessasWBcourses.
NoPerformanceDifferenceBetween
WBandTCStudents
Six recent studies (Friday, FridayStroud, Green, & Hill, 2006; Jones,
Moeeni,&Ruby,2005;Piccoli,Ahmad,
& Ives, 2001; Priluck, 2004; Scheines,
Leinhardt,Smith,&Cho,2005;Smeaton&Keogh,1999)reportedthatthere
was no student performance difference
betweenWBandTCcourses.Smeaton
and Keogh, Piccoli et al., and Jones et
al. had IT-related courses as the subject matter, whereas Priluck, Scheines
etal.,andFridayetal.useddeclarative
knowledge as the subject matter. Priluckalsoinvestigatedstudents’satisfaction,whichwashigherforTCthanfor
WBcoursesintermsofteambuilding,
criticalthinking,oralandwrittencommunications, global perspective, and
social interaction. That provides some
evidencethathighsatisfactiondoesnot
necessarily lead to high performance
inWB orTC courses. Lee (2001) also
foundthatstudents’satisfactionhadno
relation to self-reported achievement
in WB courses, which reinforces the
claim that performance, self-reported
or actual, is not necessarily related to
satisfactionwithWBcourses.
WBLearningBetterThanTCLearning
There are studies showing that studentsinWBcoursesachievebetterperformance than in TC courses (Bryan,
Campbell, & Kerr, 2003; Chou & Liu,
2005; Lockyer, Patterson, & Harper,
2001).Lockyeretal.studiedtheeffect
ofWBlearninginundergraduatehealth
care courses and concluded that WB
studentsscoredhigherthandidTCstu-
dents. Regarding student participation,
Lockyer et al. found that WB students
generated higher quality discussion by
providing more in-depth content and
references,whereasTCstudentsgenerated a higher quantity of discussion.
For Bryant et al., the subject matter
was an introductory information system course for undergraduate students.
That study concluded that WB learning was significantly better than TC
learning for concept tests, TC learning
was marginally better than WB learningforgroupproject,andWBlearning
wasjustaseffectiveasTClearningfor
activity folio. Chou and Liu measured
differences in student performance and
perception between WB learning and
TClearningforahighschoolIT-related
course.ChouandLiureportedthatWB
learningwasbetterthanTClearningfor
allvariablesforlearningeffectiveness.
TCLearningBetterThanWBLearning
Therearenotmanystudiesreporting
better performance from TC learning
compared with WB learning. Sweeney
and Ingram (2001) investigated marketing students’ perceived learning
effectiveness in WB and TC learning.
Sweeney and Ingram found that TC
learning was perceived to have higher
learning effectiveness in a tutorial setting.Bryanetal.(2003)foundthatTC
learningwasmarginallybetterthanWB
learning for group projects. Maki and
Maki (2002) compared the test scores
betweenWB andTC learning in introductorypsychologycourses,asmoderatedbystudents’comprehensionskills.
Maki and Maki determined that studentswithhighercomprehensionskills
performed better in WB than in TC
courses.However,comprehensionskills
were not significantly associated with
satisfaction,andallstudents,regardless
oftheircomprehensionskills,preferred
TCtoWBcourses.
EffectsofStudentCharacteristics
I report recent research findings of
the effects of student characteristics on
performance and perception for WB
and TC learning. Wang and Newlin’s
(2000)studyaboutpsychologystudents
revealed no student demographic differencesbetweenWBandTClearning.
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
Bryan et al.’s (2003) study determined
that academic standing (measured by
tertiary entrance score) significantly
affected performance. Alternatively,
Jonesetal.(2005)foundthatacademic
standing (as measured by GPA), age,
workhours,andpreviouscourseshadno
effectonstudents’performanceforWB
and TC learning in an undergraduate
telecommunication course. Jones et al.
didnotconsidergenderandethnicityas
predictors. As suggested by theoretical
analyses (Prinsen, Volman, & Terwel,
2007)andexperimentalresearchresults
(Arbaugh 2000; Lu, Yu, & Liu, 2003;
Wallace & Clariana, 2005; Wernet,
Olliges, & Delicath, 2000), gender and
ethnicity may have significant effects
onperformancedifferencebetweenWB
andTClearninginregardtoprocedural
knowledge, which deserves research
attention. In a study comparing performance difference between a paper test
andanonlinetestforanundergraduate
informationsystemscourse(Wallace&
Clariana), researchers discovered that
non-White female participants scored
the lowest for online tests on the midterm,butthehighestforonlinetestsfor
thefinal.However,WallaceandClariana
onlyinvestigatedthedifferencebetween
paper and online tests, not the difference between WB and TC learning. In
other words, the treatment difference
was not about WB and TC instruction,
butjustaboutthemediumfortestdelivery. When the investigation target was
WB learning only (i.e., no comparison
between WB and TC learning), Lu et
al. concluded that learning style, gender, age, job status, year of admission,
onlineexperience,andMISpreparation
hadnoeffectonstudentperformancein
an MBA or MIS course. In Lu et al.’s
study,theonlystudentcharacteristicthat
had a significant effect was ethnicity,
whichidentifiedWhitesashavingbetter
performance than Blacks inWB courses.InanotherWB-onlystudy,Arbaugh
showed that older female students may
havestrongerleaningexperiencesinWB
courses. Nevertheless, Arbaugh only
covered students’ perception, not their
actualperformance.Wernetetal.studied
socialworkstudents’satisfactioninWB
andTCcoursesandconcludedthatolder
nontraditionalstudentsfoundWBlearningmoreappealingandhadhighersat
isfaction with it.A meta-analysis study
regarding gender differences on computer-supported collaborative learning
concluded that WB courses provided a
more equitable discussion environment
in which women felt more comfortable
participating(Prinsenetal.).
Insummary,themajorityofresearch
results from existing literature, which
indicates thatWB learning is as effective as or equivalent to TC learning
forstudentperformance.However,itis
not clear for most studies whether the
subjectmatterisdeclarativeknowledge,
procedural knowledge, or a mixture of
both.WhenTClearningwasconsidered
as more effective than WB learning,
it was for perceived learning, group
project,andsatisfaction.Regardingstudent characteristics, some studies have
shownthatage(Jonesetal.,2005;Luet
al.,2003),learningstyle(Luetal.),and
priorknowledgeandskills(Jonesetal.;
Luetal.)arenotsignificantlyassociated
withperformanceinWBorTCcourses,
althoughagemaybeafactorforsatisfaction (Arbaugh, 2000; Wernet et al.,
2000).Forothercharacteristics,suchas
academicstanding,gender,andethnicity,therearemixedoruncertainresults.
Foracademicstandingasaperformance
predictor,Bryanetal.(2003)reportedit
tobesignificant,butJonesetal.reported it to be insignificant. Similarly, for
ethnicity as a performance predictor,
Lu et al. reported it to be significant,
but Wang and Newlin (2000) reported
it to be insignificant. As for gender
as a performance predictor, studies for
gendereffecthavefocusedonWebstudentsonly(Luetal.),testmediumonly
(Wallace&Clariana,2005),ordeclarative knowledge only (Scheines et al.,
2005). The effect of gender on performancedifferencebetweenWBandTC
learning for technical subjects has not
beendetermined.
StatementoftheProblem
In the present research, I investigated the effect of student characteristics
(GPA,ethnicity,gender,hitrate,andread
rate), delivery modes (WB orTC), and
evaluation methods (problem-solving
or multiple-choice questions) for the
overall performance of students (final
exam score) in technical undergradu-
atecourses.Thepredictorvariablesand
dependentvariablearefullydescribedin
theMethodsection.Thesignificanceof
this research is threefold. First, educatorsneedtounderstandtheeffectiveness
of WB learning for different students
in different subjects. The present study
sheds light on the effectiveness of WB
learning on technical subjects such as
programming and information system
development.Technical learning, by its
nature, fits the definition of procedural
knowledge. There are many aspects of
technical learning that can be difficult
to deliver by way of Web pages. For
example, debugging, as an important
skillforstudentstomasterinprogramming classes, may be more effectively taught in a face-to-face laboratory
environment. Alternatively, the logical
understandingofprogrammingfallsinto
the area of higher order skills, which
is claimed to be an effective learning
objective for WB courses (Kao, 2002).
Inthepresentstudy,Iaimedtofindthe
overalleffectivenessofWBlearningfor
technicalsubjects,whichhavebothpros
and cons for WB delivery. Second, the
presentstudyprovidesguidanceforthe
pedagogy of WB courses for technical subjects. Programming courses are
usually difficult for students to master. Understanding how different question formats can promote learning is
beneficialtoinstructors’teachingplans.
The treatment of evaluation method in
the present study provides information
about this issue. The third contribution
ofthepresentresearchisunderstanding
theeffectofGPA,gender,andethnicity
on performance difference for procedural knowledge between WB and TC
learning,whichhasnotbeenconfirmed
intheliterature.
METHOD
The present research is an empirical
study using data from an undergraduate elective programming course that I
taughtfor2yearsandapplicationdevelopment using Visual Basic. I collected
data from class records and students’
recordsintheuniversitysystem.Thepredictor variables included students’ gender, ethnicity, GPA, hit rate, read rate,
delivery mode, and evaluation method.
The dependent variable was students’
July/August2009
325
final examination scores at the end of
a semester. I used analyses of variance
(ANOVAs) and regression analyses as
thestatisticaltoolsforthisresearch.
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
Participants
Participants were 364 students from
sixWBandthreeTCcoursesin2academic years (221 WB students, 143
TC students). Tables 1–3 provide the
descriptivestatisticsforparticipants,by
ethnicity, gender, delivery method, and
evaluation method. The course was an
undergraduate elective programming
classforMISmajorsusingASP.Netto
develop Web applications. I designed
andtaughtalltheWBandTCsections.
The prerequisite for enrolling in the
targeted class was a passing grade in
an advanced Visual Basic class. The
only requirement for enrolling in WB
courses was Internet access. Students
chose freely whether to enroll in WB
orTCcourses.Duringthestudyperiod,
there were no cases in which students
couldnotenrollinaWBorTCcourse
becausethesectionwasfull.
Treatment
I designed the WB course using
WebCT. The main page of the WB
coursehastheiconsofsyllabus,assignments, PowerPoint slides, exams and
quizzes,discussionboard,e-mail,sampleprograms,labexercises,andacalendar.TheWBcoursedividedintoseven
sections. Students were advised to finish each section following a flowchart
thatoutlinedstepbystepwhattodoby
when. PowerPoint slides were annotatedbuthadnosound.Eachsectionhada
follow-upquizthatstudentshadtotake
byacertaindeadline.Follow-upquizzes
wereopen-bookandopen-note.Correct
answerstoquizzeswereavailablefrom
WebCTafterthedeadline.Becausequiz
questions were randomized from a test
bank,thechanceoftwostudentsgetting
the same questions in a quiz was slim.
Quiz records show that approximately
95%ofstudentsfinishedtheirquizzeson
time. Students accessed quiz questions
and their answers after the deadline,
eventhoughtheydidnottakethequiz.
Laboratory exercises were designed
for students to practice programming
concepts and techniques. WB students
326
JournalofEducationforBusiness
were advised to do their lab exercises
in one of the student labs any time by
a certain deadline.Although there was
no way for me to check whether students finished their laboratory exercises, that follow-up quiz questions were
partially based on laboratory exercises
providedincentiveforstudentstowork
on laboratory exercises on their own.
WB students were encouraged to email the instructor whenever they had
questionsorproblemswiththeWebCT
system.Somestudentscomplainedthat
theWebCT system did not record quiz
scores correctly or the screen froze up
after a few questions. I allowed WB
students to delete approximately 15%
ofthelowestquizscorestocompensate
for any technical problems that were
out of their control.WB students were
requiredtocomebacktotheclassroom
totakeallexaminationsinperson.
TheTCsectionofthecoursehadall
thesameteachingmaterialsastheWB
section except for a few differences.
First,IpresentedallPowerPointslides
intheclassroom,andsecond,Iadministered all quizzes in the classroom.
Quiz questions were also randomized
in the lecture section. Third, students
didtheirexercisesinalaboratoryenvironmentthatImonitored.Iwasavailabletoanswerstudentquestionswhen
theywereworkingontheirexercisesin
thelaboratory.
HypothesesforGPA,Ethnicity,
Gender,HitRate,andReadRate
This section describes GPA, ethnicity, gender, hit rate, and read rate
as the predictor variables for student
performance. The ethnicity predictor
has Black, White, Hispanic, Indian or
MiddleEastern,andAsianasvariables
in the present study. I observed that
in technical subjects, students with the
sameethnicitytendedtocollaborateand
assistoneanother.Thefactorofundergraduate classes is likely to intensify
the phenomenon of mutual assistance
because undergraduate students have
more opportunities to stay on campus
thandograduatestudents.BecauseWB
learningisnotyetcommononthecampus on which this experiment was carried out, undergraduate students who
enrolledinWBcoursesstillhadplenty
ofopportunitiestomeetoneanother.In
technicalWBcourses,inwhichstudents
havetorelyonthemselvestobeactive
learners,studentswiththesameethnicitymaydeveloptheirlearningnetwork
forcollaboration.Consideringallofthe
aforementioned issues, I hypothesized
TABLE1.DescriptiveStatistics
forCategoricalVariables(N=
364)
Variable
n
Ethnicity
Black
White
Hispanic
IndianorMiddleEastern
Asian
Gender
Male
Female
Deliverymethod
Web-based
Traditionalclassroom
Evaluationmethod
Problemsolving
Multiplechoice
12
133
40
15
164
209
155
221
143
150
214
TABLE2.Students,byDelivery
andEvaluationMethods
Delivery
method
Web-based
Traditional
classroom
Evaluationmethod
Problem
solving
Multiple
choice
110
114
40
103
TABLE3.EthnicityandGender
DistributionforWeb-based
(WB)andTraditionalClassroom(TC)Students
Variable
TC(%)
Ethnicity
Black
2.79
White
31.46
Hispanic
10.48
IndianorMiddle
Eastern
4.89
Asian
50.34
Gender
Male
56.64
Female
43.36
WB(%)
3.61
39.81
11.31
3.61
41.62
57.91
42.08
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
that ethnicity would have a significant
effect on undergraduate technical WB
learning. Regarding gender as a predictor for performance, the literature
has not provided a complete picture.
I hypothesized there would be a significant gender effect on performance
betweenWBandTCstudents.
GPAisstudents’cumulativeGPAup
tothesemestertheytooktheprogramming course under investigation. GPA
measures students’ overall academic
standing on the basis of all courses
that they have taken at the university.
Overall academic standing indicates a
student’s discipline, study habit, and
performance,whichusuallyisareliable
predictor for new academic endeavor.
HitandreadratesarevariablesforWB
students only. Hit rate is the number
ofaWBstudent’slog-onsessions,and
read rate is the number of teaching
materials a WB student retrieves to
readfromtheWBcourse.Hitandread
rates measure a student’s enthusiasm
to access course materials on theWeb.
I hypothesized that GPA, hit rate, and
readratewouldbesignificantpredictors
forperformanceinWBcourses.
HypothesisforDeliveryMode
andEvaluationMethod
The delivery mode of WB or TC
learning is a major treatment in the
present study. I hypothesized that different delivery modes would generate
differentperformance.Thepredictorof
evaluationmethod,asanothertreatment
in the present study, determines the
question format in the midterm: multiple-choice or problem-solving questions. Midterms prepare students for
theirfinalcomprehensiveexamination.
Because problem-solving questions
involve in-depth analysis of problems
and practice on solution development,
I hypothesized that problem-solving
questions would be associated with
higheroverallperformancethanwould
multiple-choicequestions.
OverallPerformanceDifference
The total score in the final comprehensive examination was used as the
dependentvariableinthepresentstudy.
The final comprehensive examination
was the same for all students in all
semesters. The final comprehensive
examination had a variety of question
formats including multiple choice,
problem solving, programming, and
concept definitions. I chose to use the
totalscoreratherthanthefinalgradeas
thedependentvariablebecausethetotal
scorewasmorecompatibleamongdifferentsectionsthanwasthefinalgrade,
which may have to be normalized in
somesections.
AnalyticalProcedure
Thefirststepinthedataanalysisprocess was to check student distribution
intermsofgender,ethnicity,GPA,and
total score betweenWB andTC learning. I applied anANOVA to determine
whether there was a significant GPA
andtotalscoredifferencebetweenWB
and TC learning. In the second step, I
applied multiple regression analyses to
theentiredatasetwithallvariables.In
thethirdstep,IappliedmultipleregressionanalysestothedatasubsetsofWB
and TC learning. In the fourth step, I
applied multiple regression analyses to
the student subgroups of low, medium,
and high GPA. All normal probability
plots for regression models showed no
sign of assumption violation and problematicresidues.
1,thepresentstudyhadasmallsample
size for Black (n = 12), Hispanic (n =
40),andIndianorMiddleEastern(n=
15) students. To check for significant
mean differences for GPA and total
score betweenWB and TC students, I
performed anANOVA. Table 4 shows
the maximums, minimums, standard
deviations, and means for GPA and
totalscore,asclassifiedbyWBandTC
learning.Themeandifferencesbetween
WBandTClearningforGPAandscore
are not significant. The insignificant
totalscoredifferencebetweenWBand
TC learning confirms those research
resultsfromtheliteratureclaimingWB
learning as equivalent to TC learning
for learning procedural knowledge.
Table5showsthedescriptivestatistics
forthevariableshitrateandreadratein
WBlearning.
GPAasSignificantPredictorfor
AllStudents,WBStudents,and
TCStudents
Table 6 shows the regression results
forallstudentsinthedataset.Model1.1
inTable6hasallthepredictorvariables.
TABLE4.DescriptiveStatistics
forGPA(Predictor)andScore
(DependentVariable)
RESULTS
Statistics
GPA Totalscore
GeneralDataDistribution
Maximum
Minimum
SD
M
MDifference
WB
TC
p
4.00
1.67
0.52
2.86
102.10
28.33
12.53
71.23
2.85
2.86
.82
71.36
71.01
.79
Tables 1–3 show the student distribution by ethnicity, gender, delivery
method, and evaluation method in the
entire data set. There were 364 studentsintheentiredatasetinwhichthe
majority was Asian (164) and White
(133).Thereweremoremale(n=209)
andWB (n = 221) students than there
werefemale(n=155)andTC(n=143)
students in the entire data set. As for
the evaluation method shown in Table
2, the data set has more students with
multiple-choice questions (n = 217)
than with problem-solving questions
(n = 150) in midterm examinations.
ComparingethnicityandgenderdistributionforWBandTCstudentsinTable
3, the main difference is the higher
percentage of White students and the
lower percentage of Asian students in
WB courses. Also, as shown in Table
Note. WB=Web-based;TC=traditional
classroom.
TABLE5.DescriptiveStatistics
forHitandReadVariablesfor
Web-basedStudents
Statistics
Maximum
Minimum
SD
M
Hit
583.00
33.00
179.34
87.62
Read
62.00
0.00
36.89
16.27
July/August2009
327
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:56 11 January 2016
BecauseGPAisthedominantpredictor,
IremoveditfromModel1.2torevealthe
significanceforothervariables.Models
1.1 and 1.2 are significant, but Model
1.2hadasmalladjustedR2(.037).Only
predictorswithasignificancelevelof≤
.05werelistedinregressionresults.In
Model1.1,whichisthemodelwithall
predictorsforallstudents,onlyGPAisa
significant predictor at the significance
level of zero, and it has a standardized
coefficientof.57.InModel1.2,which
hasallpredictorsexceptGPAforallstudents,Whitehaspositiveeffect.Inother
words,whenGPAisignored,Whitestudentstendtohavehigherscores.
Table 7 shows the regression results
forWBstudents.InModel2.1(withall
predictors),themodelissignificant,F(1)
=0,p=0.Again,GPAistheonlysignificantpredictoratasignificancelevelof
zero.TheresultsfromModel2.1(WB)
aresimilartothosefromModel1.1(all
students) with GPA as the only highly
significantandpositivepredictor.Model
2.2 (without GPA for WB students) is
alsosimilartoModel1.2(withoutGPA
forallstudents),ignoringtheBlackand
IndianorMiddleEasterneffectsbecause
oftheirsmallsamplesizes.
Table 8 shows the regression results
forTCstudents.Model3.1isthemodel
withallpredictorsforTCstudents.The
model is highly significant, F(3) = 0
(differentpredictorshavedifferentpvalues; see Table 8), with a high adjusted
R2(.462).Female,problemsolving,and
GPA are the three significant predictors, which have –.16, .267, and .612
as the standardized coefficients, respectively. The standardized coefficients in
Model 3.1 indicate the following: (a)
Female students in TC courses tend to
havelowscores;(b)problemsolvingas
theevaluationmethodisassociatedwith
highscores;and(c)GPAisstillthebest
predictor for score. Comparing Model
3.1 (all predictors for TC) with Model
2.1 (all predictors for WB students), I
noticedthatthepredictionpowerofGPA
is stronger (β = .612) in TC courses
than inWB (β = .549) courses.Whereas female and problem solving have no
significant effect on WB learning, they
respectively have significantly negative
andpositiveeffectsonTClearning.The
insignificance of female and problem
solvingonWBlearningmaybebecause
328
JournalofEducationforBusiness
oftheflexibilityoflearningpaceinWB
courses, which eliminates the learning difficulties for some students inTC
courses.InModel3.2,femaleandproblem solving are still the significant predictorsforTClearningaftertheremoval
oftheGPApredictor.
PredictorPowerofHigh,Medium,
andLowGPA
Because GPA is a highly significant
predictor, I classified students into
high-, medium-, and low-GPA groups
to perform further analyses: high GPA
≥ 3.1; medium GPA ≥ 2.5, but ≤ 3.1;
and low GPA ≤ 2.5. Table 9 shows
theregressionresultsforthehigh-GPA
(Model4.1),medium-GPA(Model4.2),
and low-GPA (Model 4.3) groups. All
models are highly significant, but only
thehigh-GPAgrouphasadecentadjustedR2(.302).Thesignificantpredictors
inthehigh-GPAgroupareAsian,problem solving, and GPA. All significant
predictors in the high GPA group have
positive standardized coefficients. In
otherwords,inthehigh-GPAgroup,the
predictorsofAsian,GPA,andproblem
solving as the evaluation method have
asignificantimpactonperformance.In
the medium-GPA group, no predictor
is significant. In the low-GPA group,
no predictor has a positive effect, but
TABLE6.RegressionResultsforAllStudents
Model
Model
significance(F)
1.1(withall
variables)
1.2(without
GPA)
df
AdjustedR2
Variable,*p
Standardized
coefficients
0.000
1
.323
GPA,0
0.001
3
.037
Black,.031
White,.008
IndianorMiddle
Eastern,.037
.57
–.113
.140
.109
*
p≤.05
TABLE7.RegressionResultsforWeb-basedStudents
Model
Model
significance(F)
2.1(withall
variables)
2.2(without
GPA)
df
AdjustedR2
Variable,*p
0.000
1
.298
GPA,0
0.004
2
.041
Black,.071
White,.012
Standardized
coefficients
.549
–.121
.17
*
p≤.05
TABLE8.RegressionResultsforTraditionalClassroomStudents
Model
Model
significance(F)
3.1(withall
variables)
3.2(without
GPA)
*
p≤.05
0.000
0.001
df
AdjustedR2
3
2
.462
.086
Variable,*p
Standardized
coefficients
Women,.011
–.160
Problemsolving,0
.267
GPA,0
.612
Women,.05
–.159
Problemsolving,.001 .264
TABLE9.RegressionResultsforAllStudents,ClassifiedbyGPA
Model
Model
significance(F) df
34.1(highGPA;
GPA≥3.1;
n=118)
4.2(medium
GPA;2.5≤
GPA