Manajemen | Fakultas Ekonomi Universitas Maritim Raja Ali Haji joeb.84.5.258-262
Journal of Education for Business
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Assessment of Student Learning: A Fast-Track
Experience
Bonnie Stivers & Jeffrey Phillips
To cite this article: Bonnie Stivers & Jeffrey Phillips (2009) Assessment of Student Learning:
A Fast-Track Experience, Journal of Education for Business, 84:5, 258-262, DOI: 10.3200/
JOEB.84.5.258-262
To link to this article: http://dx.doi.org/10.3200/JOEB.84.5.258-262
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 69
View related articles
Citing articles: 7 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 11 January 2016, At: 22:53
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
AssessmentofStudentLearning:AFastTrackExperience
BONNIESTIVERS
MOREHOUSECOLLEGE
ATLANTA,GEORGIA
ABSTRACT.Theprofessionalaccreditationagenciesaremakingdemandsfor
assessmentofstudentlearning.These
demandsarenotexpectedtolessenin
thenearfuture.Theaccreditationagenciesaresuggestingthateducatorsshould
shifttheirfocusfromwhatfacultyteach
towhatstudentslearn.Theauthorsshow
thedevelopmentandimplementationofan
assessmentofastudent-learningframework
atoneschool.Thisimplementationwas
necessarytobringtheschoolintocompliancewithnewassessmentstandardsandto
maintainaccreditationwiththeAssociation
toAdvanceCollegiateSchoolsofBusiness
International.
Keywords:assessment,Associationto
AdvanceCollegiateSchoolsofBusiness
Internationalaccreditation,assuranceof
learning,studentlearning
Copyright©2009HeldrefPublications
258
JournalofEducationforBusiness
JEFFREYPHILLIPS
CLARKATLANTAUNIVERSITY
ATLANTA,GEORGIA
T
he Association to Advance Collegiate Schools of Business
(AACSB)Internationalandotherprofessionalaccreditationagenciesareplacing
ahighpriorityonassessment.ThemessagefromtheAACSBisclear:“Ouroutputisnotteaching;itis,infact,student
learning” (2005, ¶ 2). The agency suggests that schools shift their focus from
whatfacultyteachtowhatstudentslearn.
Schoolscanmakethisshiftbyusingan
assessmentprocesstogatherdataonstudentlearningandthenusethesedatato
improvecurricularprograms.
In the present article, we describe
howthefacultymembersatoneschool
ofbusinessdevelopedanassessmentof
student learning framework during the
summer of 2005 and implemented the
plan during the 2005–2006 and 2006–
2007 academic years. This framework
was necessary to bring the school into
compliance with the new assessment
standardsandtomaintainaccreditation
with AACSB. In the first part of the
present article, we provide a brief literature review. In the second part, we
presenttheorganizationalstructureand
the model framework of general goals,
specificoutcomes,andcoursemappings
thatweredevelopedforeachofthesix
degreeprogramsinoneschoolofbusiness. In the third part of the present
article,wedescribethelessonslearned
during the school’s initial experience
withassessmentofstudentlearning.
SettingtheStage
The AACSB has adopted Palomba
andBanta’s(1999)definitionofassessment:“Assessmentisthesystematiccollection, review, and use of information
abouteducationalprogramsundertaken
for the purpose of improving student
learninganddevelopment”(p.4).
Theschoolofbusinessfacultymembers involved in the development of
the assessment framework reviewed
the AACSB’s expectations regarding
assessmentofstudentlearningandother
relevantassessmentliterature.
AACSB’sExpectationsRegarding
Assessment
The AACSB (2000) adopted “Eligibility Procedures and Standards for
BusinessAccreditation,”and1ofthe21
standards—“TheAssurance of Learning
Standards”(Standards15–21)—acknowledgesassessmentofstudentlearningasa
partofthecurriculummanagementprocess. The AACSB standards called for
schoolstodefinelearninggoalslinkedto
theirmission,assessstudentachievement
forthesegoalsforeachprogram,anduse
the assessment information to continuallyimprovetheircurriculumprograms.
Schools accredited by the AACSB
were expected to begin planning for
assessment in the 2003–2004 academic
year.TheAACSB(2005)askedschools
tofollowaspecifictransitionperiod.In
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
2003–2004—thefirst2yearsofthestandards—schools were expected to define
learning goals, translate goals into specificobjectives,andaligntheircurricula
tomeetthegoals.By2005,schoolswere
expected to have developed measurementsforeachspecificgoaltohaveand
begun collecting performance data on
atleastsomeofthemeasures.By2007,
schools should have had a complete
assurance of learning process in place,
including feedback of the assessment
dataintothecurriculum-reviewprocess.
Thetransitionperiodallowedbythe
AACSBisnowover.Startingin2007,
schools must meet the assurance-oflearning standards for accreditation.
The standards call for assessment of
student learning for each degree program in the business school. Learning
goalsmustbestated,andstudentlearning must be evaluated through direct
measures.The final step is closing the
loop and using the assessment data to
improvethecurriculum.
RelevantAssessmentLiterature
InadditiontoreviewingtheAACSB’s
expectations regarding assessment, the
faculty team at our sample school of
businessidentifiedthefollowingsourcesintheassessmentliteratureandfound
themtobeparticularlyusefulindevelopingtheassessmentframework:
1.Characteristics of effective assessment(Banta,2002);
2.Guidelines,organization,andplanning (Nichols, 1995; C.A. Palomba &
Banta,1999,2001);
3.FacultyinvolvementC.A.(Palomba
&Banta,1999);
4.Articulating goals and learning objectives (AACSB International,
2000;Bloom,1956;Erwin,1991;C.A.
Palomba&Banta,1999,2001);
5.Selecting measurement methods
andapproaches(Banta,Lund,Black,&
Oblander, 1996; Huba & Freed, 2000;
C. A. Palomba & Banta, 1999, 2001;
Walvrood,2004);
6.Examiningcurricula(C.A.Palomba
&Banta,1999);
7.Reporting and using assessment
results(Ewell,1994;C.A.Palomba&
Banta,1999,2001);
8.Assessing assessment (Romberg,
1990).
AssessmentFramework
foraSampleSchoolofBusiness
The assurance of learning standards
adoptedbytheAACSBcallforaccredited
schools of business to assess student
learningandusethefindingstoimprove
their curriculum programs. During the
springof2005,AACSBsentoursample
schoolofbusinessanoticeregardingthe
statusoftheschool’saccreditation.There
wasreasonforconcern.Bytheendofthe
first 2 years of theAACSB’s transition
period (2003–2004), the school of
business’sfacultyhadnotyetdeveloped
an assessment framework. The faculty
had to take immediate action to bring
the school of business into compliance
withtheAACSB’sassurance-of-learning
standardsandtoretainaccreditation.The
task involved putting 2 years of work
intoseveralmonths.
Weofferadescriptionoftheschool’s
efforts to organize for assessment and
present the following topics: overview,
guidelines, organization, goals framework,andmethods.
Overview
A representative body of faculty
members in the school of business
maintained the overall framework for
theassessmentofstudentlearning.This
group served in an oversight capacity.
In following curriculum management,
the faculty members of each discipline or degree program in the school
of business were expected to have an
assessmentplantocollectevidenceof
studentlearningforeachprogramthat
offeredabachelor’sormaster’sdegree.
Each academic discipline and degree
program submitted a separate report
andthefindingswereusedtodevelop,
review,orreviseeachprogramcurriculum.Theassessmentframeworkestablished the expectation that each core
course in each degree program would
passundertheviewoftheassessment
of the student-learning process over a
3-yearperiod.
GuidelinesforAssessment
ofStudentLearning
Thefacultyattheschoolofbusiness
adoptedthefollowingguidelinesforthe
assessmentofstudentlearning:
1.The purpose of assessment is to
improveeducationalprograms.According
to the AACSB’s (2000) “Curriculum
ContentandEvaluationStandard,”business
schoolsshouldsystematicallyreviewtheir
degree programs to assess effectiveness.
Program revisions should “reflect new
objectives and incorporate improvements
basedoncontemporarytheoryandpractice”
(AACSBInternational,2000,p.20).
2.Assessmentofstudentlearningand
developmentisacollaborativeprocess
involving faculty, staff, and students
(C.A.Palomba&Banta,1999).
3.Assessment is guided by the
institution’smission(C.A.Palomba&
Banta,1999).
4.Successful assessment programs
include the following: (a) identifying assessment procedures to address
all learning goals; (b) using efficient
procedures such as sampling student
work and drawing on university-wide
data where appropriate; (c) including multiple measures; (d) describing
the people, committees, and processes
involved; and (e) containing plans for
using assessment information. (N. A.
Palomba&C.A.Palomba,1999).
5.Thefacultymembersforeachacademic degree program are responsible
for the (a) identification of goals and
learning objectives, (b) selection of
measures, (c) use of assessment feedbacktomodifytheacademicprograms,
and(d)reportonassuranceoflearning
activities.
6.Assurance of learning should not
be used to make comparisons among
faculty members, departments, programs,orotherunitsintheuniversity.
7.Assurance of learning should not
be used for faculty or staff evaluation
(C.A.Palomba&Banta,1999).
8.Assuranceoflearningisasystematic,ongoing,long-termprocessofcontinuousimprovement.
9.The assessment process is evaluated(C.A.Palomba&Banta,1999).
OrganizationalStructurefor
Assessment
The organizational structure for
assessment in the school of business
included several levels: the curriculum
and assessment (CA) committee at the
school level, a CA committee for each
May/June2009
259
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
discipline or program, and an assessmentteamforeachcorecourse.
VotingmembersoftheCAcommitteefortheschoolofbusinessincluded
theassociatedeanforacademicaffairs
andonefull-timefacultymemberfrom
eachoftherespectiveacademicdisciplines. Nonvoting members included
the director of graduate business programs, the director of undergraduate business services, the assessment
coordinatorfortheschoolofbusiness,
andarepresentativefromthebusiness
community. Each elected member of
the CA committee was to serve for a
minimum of 2 consecutive academic
years. These members were elected
by popular vote by full-time faculty
membersintheirrespectiveacademic
units.ThechairpersonoftheCAcommittee was elected annually (at the
startoftheacademicyear)bypopular
vote by the voting members of the
CAcommittee.
For the fall semester of 2005, the
following specific assessment responsibilitieswereassignedtotheschoolof
businessCAcommittee:
1.Adopt guidelines for the assessmentprocess.
2.Revisetheschoolofbusinessmissionstatement.
3.Establish the discipline or programCAcommittees.
4.Plan assessment activities for the
schoolofbusiness.
5.Adopt general student-learning
goalsandlearningoutcomes.
6.Adopt discipline specific studentlearning goals and learning outcomes
using feedback from the discipline CA
committees.
7.Adopt the mapping of studentlearninggoalsandlearningoutcomesto
program requirements for each degree
program.
8.Prepare a master timeline for
implementation of the assessment processforallcorecoursesintheschoolof
business.
9.Provideoversighttothediscipline
orprogramCAcommitteesforthecollection of data, analysis, and reporting
ofassessmentfindings.
10.Review discipline or program
assessmentreports.
11.Report overall school of business
260
JournalofEducationforBusiness
assessment results to the faculty and
administrators.
12.Recommend curriculum changes to the degree programs and to the
assessment process of the school of
businessfacultyandadministrators.
13.Arrange for archiving assessment
degree plans, data, reports, curricular
revisions,andsamplesofstudentwork.
Archival data are to be retained for
6years.
Eachacademicdisciplineorprogram
was to establish a CA committee. The
discipline or program representative to
the school CA committee was to serve
as the chair of the discipline or program CA committee.The discipline or
programCAcommitteesweretodothe
following:
1.Establish the core course-assessmentteams.
2.Reviewandrevisethegeneraland
discipline-specific learning goals and
outcomes.
3.Map learning goals and outcomes
toprogramrequirements.
4.Coordinatethecollectionofassessmentdata.
5.Collect, analyze, and interpret
assessmentfindings.
6.Report assessment findings to the
school CA committee and make curriculumrecommendations.
7.Implement approved curriculum
changestothedegreeprograms.
Also at the discipline or program
level,anassessmentteamwasassigned
to each core course. Each core course
teamconsistedofatleasttwomembers:
onememberofthedisciplineorprogram
CAcommitteeandonefacultymember
withprimaryteachingresponsibilityof
thecourse.Theteamshadtheresponsibility to target core courses for assessment, identify measurement methods,
andtocollectassessmentdata.
Each faculty member in the school
ofbusinesswasexpectedtobeinvolved
in the assessment of student learning.
Every faculty member was to complete
thecurriculumrevieworaudittodevelop
the curriculum maps. In addition, each
memberwastoreviseeachcoursesyllabusinauniformformat.Coursecontent,
assignments,andtestsweretobelinked
toprogramgoalsandlearningoutcomes.
LearningGoals,Outcomes,
andMappingtoBusinessCore
Duringthesummerof2005,aconsultantandasmallfacultyteamdeveloped
a framework of general goals, specific
outcomes,andmappingforeachofthe
sixdegreeprograms:bachelorofartsin
business administration (BBA), bachelor and master of arts in accounting,
bachelorandmasterofarts,andadual
degree in accounting, MBA, and MBA
forworkingprofessionals.
DefineGeneralLearningGoals
General learning goals are broad in
natureandthereforedifficulttomeasure.
Thesegoalsdescribethedesiredcharacteristicsofthebusinessschoolgraduates.
Keeping the mission statements of the
university and the school of business in
mind,thefacultyteamidentifiedacommonsetoflearninggoalsfortheundergraduatedegreeprogramsandaslightly
different set of goals for the graduate
degreeprograms.Theteamselectedthe
following general learning goals for the
undergraduatedegreeprograms:
1.Communication: Business majors
should be literate and capable of communicatingwellinorallyandinwritten
documents.
2.Critical thinking: Business majors
should think critically, abstractly, and
logicallytoevaluateandsolvebusiness
problems.
3.Ethics: Business majors should
possess ethical values and be able to
evaluatetheethicalandsocialimpactof
businessdecisions.
4.Interaction:Businessmajorsshould
possesseffectiveinterpersonalandleadershipskills.
5.Perspectives: Business majors
should understand and consider the
effectsofforcesexternaltotheorganizationinbusinessdecisionmaking.
6.Technology: Business majors
shouldidentify,evaluate,anduseinformationtechnologytoenhancepersonal
andorganizationalproductivity.
Inadditiontoidentifyingthesegeneral
learning goals for the degree programs,
theteamidentifiedgenerallearninggoals
for each of the following disciplines:
accounting,decisionscience,economics,
finance,management,andmarketing.
IdentifySpecificLearningOutcomes
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
Specific learning outcomes stem
directlyfromthegenerallearninggoals
and are the targets of assessment. The
specificlearningoutcomesdescribeparticular actions that the graduates performandarebestexpressedwiththeuse
ofverbsinthestatements.Inmostcases,
theteamlimitedthenumberofspecific
learningoutcomestothreeforeachgenerallearninggoal.Forexample,forthe
generallearninggoalofcommunication
in the BBA, the team identified three
specificlearningoutcomes:
1.Present,discuss,anddefendviews
effectively through written and spoken
language.
2.Listeneffectively.
3.Locate, obtain, organize, report,
anduseinformationfromhuman,print,
andelectronicsources.
MapSpecificLearningOutcomes
toCoreCourses
Faculty members performed a curriculumaudittoprovidedocumentation
toidentifywhereinthecurriculumthe
learning goals and specific outcomes
were being addressed, or a curriculum
map. For this exercise, faculty completedaformforeachcorecoursethat
they taught and identified the specific
learningoutcomesthatareaddressedby
a major test, project, or paper in their
course. The collective responses were
used to develop a visual “curriculum
map” for each degree program. This
is an excellent starting point for the
understandingoftheassessmentprocess
(Gainen&Locatelli,1995,p.116).
For each specific learning outcome,
theassessmentteamselectedtwotofive
core courses and their related course
activitiesastargetsforassessmentmeasurement. Using the general learning
goalofcommunicationasanexamplein
theBBAprogram,onespecificlearning
outcome was to locate, obtain, organize, report, and use information from
human, print, and electronic sources.
Two courses that were candidates for
measurementintheassessmentprocess
and their specific activities were (a)
productionandoperationsmanagement,
in which students worked in groups to
plan, research, report, and present an
investigationofoperationsmanagement
as it applies to small business in the
economy, and (b) business policy, in
whichstudentsworkedingroupsdoing
research, analysis, and presentations
focusedonbusinesspolicy.
ApproachesandMethods
Therewereanumberofplanningquestions that needed to be addressed when
deciding on methods and approaches
to collect assessment data. The faculty
team decided on the following items
regarding the overall approach to data
collection:(a)levelsfordatacollection,
in which the faculty in the school of
businesscollectedassessmentdataatthe
university,program,andcourselevel;(b)
researchstrategies,inwhichthestrategy
was to track and compare successive
cohortsofstudents;(c)identificationof
eligible participants, in which degreeseeking students, students in a major,
andgraduatingseniorswererequiredto
participate in assessment projects; and
(d) sampling, in which faculty assessed
onlyasampleofstudentsinaneffortto
keepcostsofassessmentundercontrol.
Focusing on core courses, a representative sample of course sections was targetedforassessment.
In making decisions regarding the
selectionofdatacollectionmethods,the
faculty focused on three areas of direct
measures: tracking performance on professional licensing exams, commercially
availablestandardizedtesting,andcourseembedded testing with common exams
andcapstonecourseprojectactivities.
AssessmentExperience:
LessonsLearned
In the 2005–2006 academic year,
the school of business had developed
a promising assessment framework.
The school was reaccredited under the
AACSB and the immediate crisis was
over. The initial assessment of student
learning effort was successful for several reasons. First, there was success
because of the dean’s lead. The dean’s
office spearheaded the effort to develop an assessment framework. Second,
there was success because of funding.
The dean devoted the funds to hire a
consultant to develop the initial assuranceoflearningframework.Last,there
was success because of structure. The
schoolestablishedtheCAcommitteeat
the school level to assume assessment
responsibilities.
As the school of business moved
into the 2006–2007 academic year, the
administratorsandfacultydidnotmaintain the strong focus on assessment of
student learning as they had in 2005–
2006 academic year. A review of the
school’s experience for the 2006–2007
academic year brought to light several
issuesthatneededattention.
FacultyInvolvement
All faculty members should be
involvedintheassuranceoflearningprocess. Active involvement gives faculty
membersownershipoftheassuranceof
the learning process and provides the
experience necessary to use the assessment data to close the loop in makingcurriculumimprovements.Itmaybe
temptingtouseasmallgroupoffaculty
members to expedite the assurance-oflearning process. However, this limited
facultyinvolvementmaydefeatthebroad
purposeofassuranceoflearning.
Organization
The department and discipline committeesandteamsneedtobeorganized
and held responsible for assessment
activities. Our sample school of business established the CA committee to
assume assessment responsibilities.
An assurance of learning structure that
includes the school, department, and
disciplinelevelsdrawsonthesupportof
abroadergroupoffacultymembers.
AgreementonLearningGoals
andOutcomes
Allfacultymembersintheschoolof
business need to be involved in updatingtheframeworkoflearninggoalsand
outcomes. This framework is the basis
of the curriculum and the focus of the
assessment. The process of coming to
an agreement on these learning goals
and outcomes is a unifying experience
forfacultymembers.
Funding
The school of business should allocate adequate funds for the assessment
May/June2009
261
activities. These funds can be used to
provide faculty-release time, provide
for assessment training, develop data
collection methodologies, and fund
externaltesting.
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
DirectMeasures
The assurance of learning standards
oftheAACSBrequiretheuseofdirect
measures of student learning. The use
of indirect measures may be used to
supplement the data collected with
direct measures. One example of the
use of an indirect measure is to ask
studentsabouttheirperceptionsoftheir
learning.Another example is to collect
employer feedback on graduates’ job
performance.
ClosingtheLoop
Facultymembersmustuseassessment
data to close the loop. The administrators and faculty members must be able
to demonstrate how the assessment data
have been used to make changes in the
curricula, teaching materials, and teachingmethods.Thepurposeofassessment
istomanagethecurricularprograms.
AssesstheAssuranceoftheStudent
LearningProcess
Theassessmentprocessshouldbeevaluatedatleastonceperyear.Evaluatingthe
assessmentprocessandmakingappropriate changes in the system provides an
aspectofcontinuousimprovement.
262
JournalofEducationforBusiness
Conclusion
The present article describes the
assessment of the student-learning
framework that one school of business
facultyteamdevelopedintwosummer
months of 2005 and implemented duringthe2005–2006and2006–2007academicyears.Theassessmentframework
wasdevelopedforsixdegreeprograms.1
Thisinitialassessmentactivityinvolved
thedefinitionoflearninggoals,translation of goals into specific objectives,
alignment of curricula to meet goals,
selectionofmethods,andcollectionof
someperformancedata.
NOTES
1. The authors will provide copies of the six
degreeprogramframeworksuponrequest.
Bonnie Stivers is a professor emerita of
accounting at Kennesaw State University. She
is a visiting professor of accounting at MorehouseCollege.Herareaofinterestismanagerial
accounting.
Jeffrey Phillips’s teaching interests include
accounting information systems and cost management. His research interests include internal
control systems, cost management systems, and
corporategovernanceandcompliance.
Correspondence concerning this article should
be addressed to Bonnie Stivers, 6935 Cordery
Road,Cumming,GA30040,USA.
E-mail:[email protected]
REFERENCES
Association to Advance Collegiate Schools of
Business(AACSB)International.(2000,April).
Achievingqualityandcontinuousimprovement
through self-evaluation and peer review: Standardsforaccreditation.St.Louis,MO:Author.
AACSB International. (2005). Overview of
assessment. Retrieved August 1, 2005, from
http://www.aacsb.edu/resourcecenters/assessment/overview-expectations.asp
Banta,T.W.(Ed.).(2002).Buildingascholarship
ofassessment.SanFrancisco:Jossey-Bass.
Banta,T.W.,Lund,J.P.,Black,K.E.,&Oblander, F.W. (1996). Assessment in practice: Putting principles to work on college campuses.
SanFrancisco:Jossey-Bass.
Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals handbook. Part 1: Cognitive
domain.WhitePlains,NY:Longman.
Erwin, T. D. (1991). Assessing student learning
and development: A guide to the principles,
goals,andmethodsofdeterminingcollegeoutcomes.SanFrancisco:Jossey-Bass.
Ewell, P.T. (1994). A policy guide for assessment: Making good use of tasks in critical
thinking.Princeton,NJ:EducationalTesting
Service.
Gainen,J.,&Locatelli,P.(1995).Assessmentfor
the new curriculum: A guide for professional
accounting programs. Sarasota, FL:American
AccountingAssociation.
Huba,M.E.,&Freed,J.E.(2000).Learner-centeredassessmentoncollegecampuses:Shifting
the focus from teaching to learning. Needham
Heights,MA:Allyn&Bacon.
Nichols, J. O. (1995). A practioner’s handbook
for institutional effectiveness and student outcomes assessment implementation. NewYork:
AgathonPress.
Palomba, C.A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and
improvingassessmentinhighereducation.San
Francisco:Jossey-Bass.
Palomba, C. A., & Banta, T. W. (Eds.). (2001).
Assessing student competence in accredited
disciplines.Sterling,VA:StylusPublishing.
Palomba, N.A., & Palomba, C.A. (1999, May/
June). AACSB accreditation and assessment
inBallStateUniversity’sCollegeofBusiness.
AssessmentUpdate,11(3),4–15.
Romberg,E.(Ed.).(1990).Outcomesassessment:
A resource book. Washington, DC: American
AssociationofDentalSchools.
Walvoord, B. E. (2004). Assessment clear and
simple: A practical guide for institutions,
departments,andgeneraleducation.SanFrancisco:Jossey-Bass.
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Assessment of Student Learning: A Fast-Track
Experience
Bonnie Stivers & Jeffrey Phillips
To cite this article: Bonnie Stivers & Jeffrey Phillips (2009) Assessment of Student Learning:
A Fast-Track Experience, Journal of Education for Business, 84:5, 258-262, DOI: 10.3200/
JOEB.84.5.258-262
To link to this article: http://dx.doi.org/10.3200/JOEB.84.5.258-262
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 69
View related articles
Citing articles: 7 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 11 January 2016, At: 22:53
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
AssessmentofStudentLearning:AFastTrackExperience
BONNIESTIVERS
MOREHOUSECOLLEGE
ATLANTA,GEORGIA
ABSTRACT.Theprofessionalaccreditationagenciesaremakingdemandsfor
assessmentofstudentlearning.These
demandsarenotexpectedtolessenin
thenearfuture.Theaccreditationagenciesaresuggestingthateducatorsshould
shifttheirfocusfromwhatfacultyteach
towhatstudentslearn.Theauthorsshow
thedevelopmentandimplementationofan
assessmentofastudent-learningframework
atoneschool.Thisimplementationwas
necessarytobringtheschoolintocompliancewithnewassessmentstandardsandto
maintainaccreditationwiththeAssociation
toAdvanceCollegiateSchoolsofBusiness
International.
Keywords:assessment,Associationto
AdvanceCollegiateSchoolsofBusiness
Internationalaccreditation,assuranceof
learning,studentlearning
Copyright©2009HeldrefPublications
258
JournalofEducationforBusiness
JEFFREYPHILLIPS
CLARKATLANTAUNIVERSITY
ATLANTA,GEORGIA
T
he Association to Advance Collegiate Schools of Business
(AACSB)Internationalandotherprofessionalaccreditationagenciesareplacing
ahighpriorityonassessment.ThemessagefromtheAACSBisclear:“Ouroutputisnotteaching;itis,infact,student
learning” (2005, ¶ 2). The agency suggests that schools shift their focus from
whatfacultyteachtowhatstudentslearn.
Schoolscanmakethisshiftbyusingan
assessmentprocesstogatherdataonstudentlearningandthenusethesedatato
improvecurricularprograms.
In the present article, we describe
howthefacultymembersatoneschool
ofbusinessdevelopedanassessmentof
student learning framework during the
summer of 2005 and implemented the
plan during the 2005–2006 and 2006–
2007 academic years. This framework
was necessary to bring the school into
compliance with the new assessment
standardsandtomaintainaccreditation
with AACSB. In the first part of the
present article, we provide a brief literature review. In the second part, we
presenttheorganizationalstructureand
the model framework of general goals,
specificoutcomes,andcoursemappings
thatweredevelopedforeachofthesix
degreeprogramsinoneschoolofbusiness. In the third part of the present
article,wedescribethelessonslearned
during the school’s initial experience
withassessmentofstudentlearning.
SettingtheStage
The AACSB has adopted Palomba
andBanta’s(1999)definitionofassessment:“Assessmentisthesystematiccollection, review, and use of information
abouteducationalprogramsundertaken
for the purpose of improving student
learninganddevelopment”(p.4).
Theschoolofbusinessfacultymembers involved in the development of
the assessment framework reviewed
the AACSB’s expectations regarding
assessmentofstudentlearningandother
relevantassessmentliterature.
AACSB’sExpectationsRegarding
Assessment
The AACSB (2000) adopted “Eligibility Procedures and Standards for
BusinessAccreditation,”and1ofthe21
standards—“TheAssurance of Learning
Standards”(Standards15–21)—acknowledgesassessmentofstudentlearningasa
partofthecurriculummanagementprocess. The AACSB standards called for
schoolstodefinelearninggoalslinkedto
theirmission,assessstudentachievement
forthesegoalsforeachprogram,anduse
the assessment information to continuallyimprovetheircurriculumprograms.
Schools accredited by the AACSB
were expected to begin planning for
assessment in the 2003–2004 academic
year.TheAACSB(2005)askedschools
tofollowaspecifictransitionperiod.In
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
2003–2004—thefirst2yearsofthestandards—schools were expected to define
learning goals, translate goals into specificobjectives,andaligntheircurricula
tomeetthegoals.By2005,schoolswere
expected to have developed measurementsforeachspecificgoaltohaveand
begun collecting performance data on
atleastsomeofthemeasures.By2007,
schools should have had a complete
assurance of learning process in place,
including feedback of the assessment
dataintothecurriculum-reviewprocess.
Thetransitionperiodallowedbythe
AACSBisnowover.Startingin2007,
schools must meet the assurance-oflearning standards for accreditation.
The standards call for assessment of
student learning for each degree program in the business school. Learning
goalsmustbestated,andstudentlearning must be evaluated through direct
measures.The final step is closing the
loop and using the assessment data to
improvethecurriculum.
RelevantAssessmentLiterature
InadditiontoreviewingtheAACSB’s
expectations regarding assessment, the
faculty team at our sample school of
businessidentifiedthefollowingsourcesintheassessmentliteratureandfound
themtobeparticularlyusefulindevelopingtheassessmentframework:
1.Characteristics of effective assessment(Banta,2002);
2.Guidelines,organization,andplanning (Nichols, 1995; C.A. Palomba &
Banta,1999,2001);
3.FacultyinvolvementC.A.(Palomba
&Banta,1999);
4.Articulating goals and learning objectives (AACSB International,
2000;Bloom,1956;Erwin,1991;C.A.
Palomba&Banta,1999,2001);
5.Selecting measurement methods
andapproaches(Banta,Lund,Black,&
Oblander, 1996; Huba & Freed, 2000;
C. A. Palomba & Banta, 1999, 2001;
Walvrood,2004);
6.Examiningcurricula(C.A.Palomba
&Banta,1999);
7.Reporting and using assessment
results(Ewell,1994;C.A.Palomba&
Banta,1999,2001);
8.Assessing assessment (Romberg,
1990).
AssessmentFramework
foraSampleSchoolofBusiness
The assurance of learning standards
adoptedbytheAACSBcallforaccredited
schools of business to assess student
learningandusethefindingstoimprove
their curriculum programs. During the
springof2005,AACSBsentoursample
schoolofbusinessanoticeregardingthe
statusoftheschool’saccreditation.There
wasreasonforconcern.Bytheendofthe
first 2 years of theAACSB’s transition
period (2003–2004), the school of
business’sfacultyhadnotyetdeveloped
an assessment framework. The faculty
had to take immediate action to bring
the school of business into compliance
withtheAACSB’sassurance-of-learning
standardsandtoretainaccreditation.The
task involved putting 2 years of work
intoseveralmonths.
Weofferadescriptionoftheschool’s
efforts to organize for assessment and
present the following topics: overview,
guidelines, organization, goals framework,andmethods.
Overview
A representative body of faculty
members in the school of business
maintained the overall framework for
theassessmentofstudentlearning.This
group served in an oversight capacity.
In following curriculum management,
the faculty members of each discipline or degree program in the school
of business were expected to have an
assessmentplantocollectevidenceof
studentlearningforeachprogramthat
offeredabachelor’sormaster’sdegree.
Each academic discipline and degree
program submitted a separate report
andthefindingswereusedtodevelop,
review,orreviseeachprogramcurriculum.Theassessmentframeworkestablished the expectation that each core
course in each degree program would
passundertheviewoftheassessment
of the student-learning process over a
3-yearperiod.
GuidelinesforAssessment
ofStudentLearning
Thefacultyattheschoolofbusiness
adoptedthefollowingguidelinesforthe
assessmentofstudentlearning:
1.The purpose of assessment is to
improveeducationalprograms.According
to the AACSB’s (2000) “Curriculum
ContentandEvaluationStandard,”business
schoolsshouldsystematicallyreviewtheir
degree programs to assess effectiveness.
Program revisions should “reflect new
objectives and incorporate improvements
basedoncontemporarytheoryandpractice”
(AACSBInternational,2000,p.20).
2.Assessmentofstudentlearningand
developmentisacollaborativeprocess
involving faculty, staff, and students
(C.A.Palomba&Banta,1999).
3.Assessment is guided by the
institution’smission(C.A.Palomba&
Banta,1999).
4.Successful assessment programs
include the following: (a) identifying assessment procedures to address
all learning goals; (b) using efficient
procedures such as sampling student
work and drawing on university-wide
data where appropriate; (c) including multiple measures; (d) describing
the people, committees, and processes
involved; and (e) containing plans for
using assessment information. (N. A.
Palomba&C.A.Palomba,1999).
5.Thefacultymembersforeachacademic degree program are responsible
for the (a) identification of goals and
learning objectives, (b) selection of
measures, (c) use of assessment feedbacktomodifytheacademicprograms,
and(d)reportonassuranceoflearning
activities.
6.Assurance of learning should not
be used to make comparisons among
faculty members, departments, programs,orotherunitsintheuniversity.
7.Assurance of learning should not
be used for faculty or staff evaluation
(C.A.Palomba&Banta,1999).
8.Assuranceoflearningisasystematic,ongoing,long-termprocessofcontinuousimprovement.
9.The assessment process is evaluated(C.A.Palomba&Banta,1999).
OrganizationalStructurefor
Assessment
The organizational structure for
assessment in the school of business
included several levels: the curriculum
and assessment (CA) committee at the
school level, a CA committee for each
May/June2009
259
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
discipline or program, and an assessmentteamforeachcorecourse.
VotingmembersoftheCAcommitteefortheschoolofbusinessincluded
theassociatedeanforacademicaffairs
andonefull-timefacultymemberfrom
eachoftherespectiveacademicdisciplines. Nonvoting members included
the director of graduate business programs, the director of undergraduate business services, the assessment
coordinatorfortheschoolofbusiness,
andarepresentativefromthebusiness
community. Each elected member of
the CA committee was to serve for a
minimum of 2 consecutive academic
years. These members were elected
by popular vote by full-time faculty
membersintheirrespectiveacademic
units.ThechairpersonoftheCAcommittee was elected annually (at the
startoftheacademicyear)bypopular
vote by the voting members of the
CAcommittee.
For the fall semester of 2005, the
following specific assessment responsibilitieswereassignedtotheschoolof
businessCAcommittee:
1.Adopt guidelines for the assessmentprocess.
2.Revisetheschoolofbusinessmissionstatement.
3.Establish the discipline or programCAcommittees.
4.Plan assessment activities for the
schoolofbusiness.
5.Adopt general student-learning
goalsandlearningoutcomes.
6.Adopt discipline specific studentlearning goals and learning outcomes
using feedback from the discipline CA
committees.
7.Adopt the mapping of studentlearninggoalsandlearningoutcomesto
program requirements for each degree
program.
8.Prepare a master timeline for
implementation of the assessment processforallcorecoursesintheschoolof
business.
9.Provideoversighttothediscipline
orprogramCAcommitteesforthecollection of data, analysis, and reporting
ofassessmentfindings.
10.Review discipline or program
assessmentreports.
11.Report overall school of business
260
JournalofEducationforBusiness
assessment results to the faculty and
administrators.
12.Recommend curriculum changes to the degree programs and to the
assessment process of the school of
businessfacultyandadministrators.
13.Arrange for archiving assessment
degree plans, data, reports, curricular
revisions,andsamplesofstudentwork.
Archival data are to be retained for
6years.
Eachacademicdisciplineorprogram
was to establish a CA committee. The
discipline or program representative to
the school CA committee was to serve
as the chair of the discipline or program CA committee.The discipline or
programCAcommitteesweretodothe
following:
1.Establish the core course-assessmentteams.
2.Reviewandrevisethegeneraland
discipline-specific learning goals and
outcomes.
3.Map learning goals and outcomes
toprogramrequirements.
4.Coordinatethecollectionofassessmentdata.
5.Collect, analyze, and interpret
assessmentfindings.
6.Report assessment findings to the
school CA committee and make curriculumrecommendations.
7.Implement approved curriculum
changestothedegreeprograms.
Also at the discipline or program
level,anassessmentteamwasassigned
to each core course. Each core course
teamconsistedofatleasttwomembers:
onememberofthedisciplineorprogram
CAcommitteeandonefacultymember
withprimaryteachingresponsibilityof
thecourse.Theteamshadtheresponsibility to target core courses for assessment, identify measurement methods,
andtocollectassessmentdata.
Each faculty member in the school
ofbusinesswasexpectedtobeinvolved
in the assessment of student learning.
Every faculty member was to complete
thecurriculumrevieworaudittodevelop
the curriculum maps. In addition, each
memberwastoreviseeachcoursesyllabusinauniformformat.Coursecontent,
assignments,andtestsweretobelinked
toprogramgoalsandlearningoutcomes.
LearningGoals,Outcomes,
andMappingtoBusinessCore
Duringthesummerof2005,aconsultantandasmallfacultyteamdeveloped
a framework of general goals, specific
outcomes,andmappingforeachofthe
sixdegreeprograms:bachelorofartsin
business administration (BBA), bachelor and master of arts in accounting,
bachelorandmasterofarts,andadual
degree in accounting, MBA, and MBA
forworkingprofessionals.
DefineGeneralLearningGoals
General learning goals are broad in
natureandthereforedifficulttomeasure.
Thesegoalsdescribethedesiredcharacteristicsofthebusinessschoolgraduates.
Keeping the mission statements of the
university and the school of business in
mind,thefacultyteamidentifiedacommonsetoflearninggoalsfortheundergraduatedegreeprogramsandaslightly
different set of goals for the graduate
degreeprograms.Theteamselectedthe
following general learning goals for the
undergraduatedegreeprograms:
1.Communication: Business majors
should be literate and capable of communicatingwellinorallyandinwritten
documents.
2.Critical thinking: Business majors
should think critically, abstractly, and
logicallytoevaluateandsolvebusiness
problems.
3.Ethics: Business majors should
possess ethical values and be able to
evaluatetheethicalandsocialimpactof
businessdecisions.
4.Interaction:Businessmajorsshould
possesseffectiveinterpersonalandleadershipskills.
5.Perspectives: Business majors
should understand and consider the
effectsofforcesexternaltotheorganizationinbusinessdecisionmaking.
6.Technology: Business majors
shouldidentify,evaluate,anduseinformationtechnologytoenhancepersonal
andorganizationalproductivity.
Inadditiontoidentifyingthesegeneral
learning goals for the degree programs,
theteamidentifiedgenerallearninggoals
for each of the following disciplines:
accounting,decisionscience,economics,
finance,management,andmarketing.
IdentifySpecificLearningOutcomes
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
Specific learning outcomes stem
directlyfromthegenerallearninggoals
and are the targets of assessment. The
specificlearningoutcomesdescribeparticular actions that the graduates performandarebestexpressedwiththeuse
ofverbsinthestatements.Inmostcases,
theteamlimitedthenumberofspecific
learningoutcomestothreeforeachgenerallearninggoal.Forexample,forthe
generallearninggoalofcommunication
in the BBA, the team identified three
specificlearningoutcomes:
1.Present,discuss,anddefendviews
effectively through written and spoken
language.
2.Listeneffectively.
3.Locate, obtain, organize, report,
anduseinformationfromhuman,print,
andelectronicsources.
MapSpecificLearningOutcomes
toCoreCourses
Faculty members performed a curriculumaudittoprovidedocumentation
toidentifywhereinthecurriculumthe
learning goals and specific outcomes
were being addressed, or a curriculum
map. For this exercise, faculty completedaformforeachcorecoursethat
they taught and identified the specific
learningoutcomesthatareaddressedby
a major test, project, or paper in their
course. The collective responses were
used to develop a visual “curriculum
map” for each degree program. This
is an excellent starting point for the
understandingoftheassessmentprocess
(Gainen&Locatelli,1995,p.116).
For each specific learning outcome,
theassessmentteamselectedtwotofive
core courses and their related course
activitiesastargetsforassessmentmeasurement. Using the general learning
goalofcommunicationasanexamplein
theBBAprogram,onespecificlearning
outcome was to locate, obtain, organize, report, and use information from
human, print, and electronic sources.
Two courses that were candidates for
measurementintheassessmentprocess
and their specific activities were (a)
productionandoperationsmanagement,
in which students worked in groups to
plan, research, report, and present an
investigationofoperationsmanagement
as it applies to small business in the
economy, and (b) business policy, in
whichstudentsworkedingroupsdoing
research, analysis, and presentations
focusedonbusinesspolicy.
ApproachesandMethods
Therewereanumberofplanningquestions that needed to be addressed when
deciding on methods and approaches
to collect assessment data. The faculty
team decided on the following items
regarding the overall approach to data
collection:(a)levelsfordatacollection,
in which the faculty in the school of
businesscollectedassessmentdataatthe
university,program,andcourselevel;(b)
researchstrategies,inwhichthestrategy
was to track and compare successive
cohortsofstudents;(c)identificationof
eligible participants, in which degreeseeking students, students in a major,
andgraduatingseniorswererequiredto
participate in assessment projects; and
(d) sampling, in which faculty assessed
onlyasampleofstudentsinaneffortto
keepcostsofassessmentundercontrol.
Focusing on core courses, a representative sample of course sections was targetedforassessment.
In making decisions regarding the
selectionofdatacollectionmethods,the
faculty focused on three areas of direct
measures: tracking performance on professional licensing exams, commercially
availablestandardizedtesting,andcourseembedded testing with common exams
andcapstonecourseprojectactivities.
AssessmentExperience:
LessonsLearned
In the 2005–2006 academic year,
the school of business had developed
a promising assessment framework.
The school was reaccredited under the
AACSB and the immediate crisis was
over. The initial assessment of student
learning effort was successful for several reasons. First, there was success
because of the dean’s lead. The dean’s
office spearheaded the effort to develop an assessment framework. Second,
there was success because of funding.
The dean devoted the funds to hire a
consultant to develop the initial assuranceoflearningframework.Last,there
was success because of structure. The
schoolestablishedtheCAcommitteeat
the school level to assume assessment
responsibilities.
As the school of business moved
into the 2006–2007 academic year, the
administratorsandfacultydidnotmaintain the strong focus on assessment of
student learning as they had in 2005–
2006 academic year. A review of the
school’s experience for the 2006–2007
academic year brought to light several
issuesthatneededattention.
FacultyInvolvement
All faculty members should be
involvedintheassuranceoflearningprocess. Active involvement gives faculty
membersownershipoftheassuranceof
the learning process and provides the
experience necessary to use the assessment data to close the loop in makingcurriculumimprovements.Itmaybe
temptingtouseasmallgroupoffaculty
members to expedite the assurance-oflearning process. However, this limited
facultyinvolvementmaydefeatthebroad
purposeofassuranceoflearning.
Organization
The department and discipline committeesandteamsneedtobeorganized
and held responsible for assessment
activities. Our sample school of business established the CA committee to
assume assessment responsibilities.
An assurance of learning structure that
includes the school, department, and
disciplinelevelsdrawsonthesupportof
abroadergroupoffacultymembers.
AgreementonLearningGoals
andOutcomes
Allfacultymembersintheschoolof
business need to be involved in updatingtheframeworkoflearninggoalsand
outcomes. This framework is the basis
of the curriculum and the focus of the
assessment. The process of coming to
an agreement on these learning goals
and outcomes is a unifying experience
forfacultymembers.
Funding
The school of business should allocate adequate funds for the assessment
May/June2009
261
activities. These funds can be used to
provide faculty-release time, provide
for assessment training, develop data
collection methodologies, and fund
externaltesting.
Downloaded by [Universitas Maritim Raja Ali Haji] at 22:53 11 January 2016
DirectMeasures
The assurance of learning standards
oftheAACSBrequiretheuseofdirect
measures of student learning. The use
of indirect measures may be used to
supplement the data collected with
direct measures. One example of the
use of an indirect measure is to ask
studentsabouttheirperceptionsoftheir
learning.Another example is to collect
employer feedback on graduates’ job
performance.
ClosingtheLoop
Facultymembersmustuseassessment
data to close the loop. The administrators and faculty members must be able
to demonstrate how the assessment data
have been used to make changes in the
curricula, teaching materials, and teachingmethods.Thepurposeofassessment
istomanagethecurricularprograms.
AssesstheAssuranceoftheStudent
LearningProcess
Theassessmentprocessshouldbeevaluatedatleastonceperyear.Evaluatingthe
assessmentprocessandmakingappropriate changes in the system provides an
aspectofcontinuousimprovement.
262
JournalofEducationforBusiness
Conclusion
The present article describes the
assessment of the student-learning
framework that one school of business
facultyteamdevelopedintwosummer
months of 2005 and implemented duringthe2005–2006and2006–2007academicyears.Theassessmentframework
wasdevelopedforsixdegreeprograms.1
Thisinitialassessmentactivityinvolved
thedefinitionoflearninggoals,translation of goals into specific objectives,
alignment of curricula to meet goals,
selectionofmethods,andcollectionof
someperformancedata.
NOTES
1. The authors will provide copies of the six
degreeprogramframeworksuponrequest.
Bonnie Stivers is a professor emerita of
accounting at Kennesaw State University. She
is a visiting professor of accounting at MorehouseCollege.Herareaofinterestismanagerial
accounting.
Jeffrey Phillips’s teaching interests include
accounting information systems and cost management. His research interests include internal
control systems, cost management systems, and
corporategovernanceandcompliance.
Correspondence concerning this article should
be addressed to Bonnie Stivers, 6935 Cordery
Road,Cumming,GA30040,USA.
E-mail:[email protected]
REFERENCES
Association to Advance Collegiate Schools of
Business(AACSB)International.(2000,April).
Achievingqualityandcontinuousimprovement
through self-evaluation and peer review: Standardsforaccreditation.St.Louis,MO:Author.
AACSB International. (2005). Overview of
assessment. Retrieved August 1, 2005, from
http://www.aacsb.edu/resourcecenters/assessment/overview-expectations.asp
Banta,T.W.(Ed.).(2002).Buildingascholarship
ofassessment.SanFrancisco:Jossey-Bass.
Banta,T.W.,Lund,J.P.,Black,K.E.,&Oblander, F.W. (1996). Assessment in practice: Putting principles to work on college campuses.
SanFrancisco:Jossey-Bass.
Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives: The classification of educational goals handbook. Part 1: Cognitive
domain.WhitePlains,NY:Longman.
Erwin, T. D. (1991). Assessing student learning
and development: A guide to the principles,
goals,andmethodsofdeterminingcollegeoutcomes.SanFrancisco:Jossey-Bass.
Ewell, P.T. (1994). A policy guide for assessment: Making good use of tasks in critical
thinking.Princeton,NJ:EducationalTesting
Service.
Gainen,J.,&Locatelli,P.(1995).Assessmentfor
the new curriculum: A guide for professional
accounting programs. Sarasota, FL:American
AccountingAssociation.
Huba,M.E.,&Freed,J.E.(2000).Learner-centeredassessmentoncollegecampuses:Shifting
the focus from teaching to learning. Needham
Heights,MA:Allyn&Bacon.
Nichols, J. O. (1995). A practioner’s handbook
for institutional effectiveness and student outcomes assessment implementation. NewYork:
AgathonPress.
Palomba, C.A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and
improvingassessmentinhighereducation.San
Francisco:Jossey-Bass.
Palomba, C. A., & Banta, T. W. (Eds.). (2001).
Assessing student competence in accredited
disciplines.Sterling,VA:StylusPublishing.
Palomba, N.A., & Palomba, C.A. (1999, May/
June). AACSB accreditation and assessment
inBallStateUniversity’sCollegeofBusiness.
AssessmentUpdate,11(3),4–15.
Romberg,E.(Ed.).(1990).Outcomesassessment:
A resource book. Washington, DC: American
AssociationofDentalSchools.
Walvoord, B. E. (2004). Assessment clear and
simple: A practical guide for institutions,
departments,andgeneraleducation.SanFrancisco:Jossey-Bass.