PRE AND POST TESTING

Robert Newton The following recipe describes a test methodology currently being used to test a group of postgraduate students undertaking The Robert Gordon University

a programme of instruction in bibliographic classification, but which has broad application possibilities (see the overview in the information page).

Use ◆ Evaluation of 'established' CAL packages i.e. those in which issues related to content and interface design have

already been tested and proven to be acceptable.

If you intend that pre and

Process

post testing of students

1 . Decide on the composition of the test group

should be complemented by Ideally, to perform pre and post testing, you should aim to achieve a student test group of at least 30 students. You will find other evaluation techniques

that should you try to use this technique on very large student populations the work involved in determination of student

e.g. paired testing of profiles and analysis of test results becomes unmanageable (unless you are putting considerable staffing resources into students, you should aim to

performing the survey).

use a test group of 60 students in order to allow you to make reliable statistical conclusions from your data.

2 . Create your student profiles

Pre and post testing is When creating your student profiles, try to take into account the various external factors which can influence student extremely sensitive to the learning. Profiles should therefore be created to elicit detail of: external factors which

◆ motivational factors,

influence student learning. It ◆ personal factors (derived from interview), would be very wrong to

assume that we can discount ◆ educational factors (including previous experience and use of computers and CAL),

a variety of external factors

◆ learning style.

which will affect student performance when considering the reasons for

3 . Familiarisation with CAL package

variation between pre and post test responses.

Although a package might be simple to use it is important to ensure that students are familiar with all aspects of how to use the various features such as taking notes online, activating the glossary, navigating through various pages and beginning an online test session.

Allow time to make sure that

You could consider organising a familiarisation session prior to your evaluation.

all are comfortable using the CAL package.

Have a question at this point As part of the evaluation session, students could be asked to complete a questionnaire which includes questions relating which asks the student if he/ to their degree of comfort and confidence in being able to use the CAL package.

she would be prepared to use CAL on occasion as a substitute for lectures.

In order that students have a

4 . Pre and post testing

basic foundation of the

A . work around the CAL package

prerequisite subject content, run the CAL session after the

Decide when you want to pre and post test your students. Think about how much of the subject content they need to delivery of a lecture on an

know before a pre test. It is advisable to post test immediately after they have completed their study of the material in the appropriate topic as one part

CAL package. of their formal programme of

OR study.

B selection of groups for two alternative modes of learning One option might be to use the data derived from student profiles and responses to stage 3 questions so that the cohort

can be divided into two. One group can use the CAL package as a substitute for lectures (on at least 2 occasions). The second group can follow the standard lecture programme. Both groups should undertake pre and post tests.

C work around the lecture This is the reverse of Stage A. At this stage all students take the CAL unit prior to the delivery of the lecture in the topic.

The pre and post testing is delivered immediately prior to and immediately after the lecture. These tests could be online or paper-based.

Ideally there should be some

5 . Analysis of results

form of tracking and timing The various tests will provide a huge amount of data some of it will be raw numeric data that can be analysed using

mechanism built into the standard statistical tests. package being used and this

will be very valuable when The advantage of having taken such an exhaustive approach to examining all the factors which might affect student attempting when analysing learning (when building up the student profiles) is that, as was said earlier there are a large number of variables which

results of tests. can potentially affect student learning. At the analysis stage you will be able to take account of all of these variables.

Other Relevant Pages Recipes

Much of this seems to be very complex and intensive of staff time and effort. Pre and post testing of students is not Questionnaires

an easy evaluation strategy, but it does provide a great deal of useful information on how students learn. Checklists

Confidence logs Interviews Designing experiments Cost effectiveness System log data Observation techniques

Information pages Learning styles

Pre and post testing Statistics questions Student sample Pedagogic toolkit Working with groups

Serving suggestions

PRE AND POST TESTING