Set Datang di SMAN 8 Batam Analysing data

PREST
Practitioner Research and
Evaluation Skills Training in
Open and Distance Learning

MODULE

Getting and analysing
quantitative data

A3

The PREST training resources aim to help open and distance learning practitioners develop
and extend their research and evaluation skills.They can be used on a self-study basis or by
training providers.The resources consist of two sets of materials: a six-module foundation
course in research and evaluation skills and six handbooks in specific research areas of
ODL.There is an accompanying user guide. A full list appears on the back cover.
The print-based materials are freely downloadable from the Commonwealth of Learning
(COL) website (www.col.org/prest). Providers wishing to print and bind copies can apply
for camera-ready copy which includes colour covers (info@col.org).They were developed
by the International Research Foundation for Open Learning (www.irfol.ac.uk) on behalf

of COL.

The PREST core team
Charlotte Creed (Programme coordinator)
Richard Freeman (Instructional designer, editor and author)
Bernadette Robinson (Academic editor and author)
Alan Woodley (Academic editor and author)

Additional members
Terry Allsop (Critical reviewer)
Alicia Fentiman (Basic education adviser)
Graham Hiles (Page layout)
Helen Lentell (Commonwealth of Learning Training Programme Manager)
Santosh Panda (External academic editor)
Reehana Raza (Higher education adviser)

Steering group
The PREST programme has been guided by a distinguished international steering group
including: Peter Cookson, Raj Dhanarajan,Tony Dodds,Terry Evans, Olugbemiro Jegede,
David Murphy, Evie Nonyongo, Santosh Panda and Hilary Perraton.

Acknowledgements
We are particularly grateful to Hilary Perraton and Raj Dhanarajan who originally
conceived of the PREST programme and have supported the project throughout. Among
those to whom we are indebted for support, information and ideas are Honor Carter, Kate
Crofts, John Daniel, Nick Gao, Jenny Glennie, Keith Harry, Colin Latchem, Lydia Meister,
Roger Mills, Sanjaya Mishra, Ros Morpeth, Rod Tyrer, Paul West and Dave Wilson. In
developing the materials, we have drawn inspiration from the lead provided by Roger
Mitton in his handbook, Mitton, R. 1982 Practical research in distance education, Cambridge:
International Extension College.

Handbook A3: Getting and analysing quantitative data
Author: Alan Woodley
Critical reviewers: Richard Freeman, Santosh Panda and Bernadette Robinson.
© 2004 Commonwealth of Learning
ISBN 1-894975-13-8
Permission is granted for use by third parties on condition that attribution to COL is
retained and that their use is strictly for non-commercial purposes and not for resale.
Training providers wishing to version the materials must follow COL's rules on copyright
matters.

Permissions
See the last page of the module.

Contents
Getting and analysing quantitative data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
Aims of the module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1
Module objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2
Module organisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2
Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4
Unit 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Unit overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
The rules of quantitative methods and how to apply them: an introductory case study . . . . .6
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Feedback to selected activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
Unit 2:What do we mean by quantitative methods? . . . . . . . . . . . . . . . . . . . . . .17
Unit overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Which questions can be answered with a quantitative approach? . . . . . . . . . . . . . . . . . . . . . . .18
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
The rest of the module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21
Unit 3: Analysing other people’s data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Unit overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
The data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23
Raw numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24
Percentages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Excel for beginners: calculating totals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26
The open schools case study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33
Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45
Feedback to selected activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46
Unit 4: Quantitative institutional data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Unit overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56

Types of data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56
Types of institutional data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59
Dealing with quantitative data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59
Summarising . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60
Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65
Spread, dispersion and deviation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68
Are we getting more young students? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73
Patterns and trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86
Feedback to selected activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87
Unit 5: Doing institutional research ‘from scratch’ . . . . . . . . . . . . . . . . . . . . . . . .89
Unit overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89
Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90
Validity and reliability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90

Commonwealth of Learning

iii

Module A3 – Getting and analysing quantitative data

The dimensions of data collection methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
The range of quantitative research methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .94
Designing good questions to ask people . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .94
Guidelines for good question writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97
Forms of questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100
Designing good questionnaires . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .108
Designing for disability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113
Carrying out a survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .125
Feedback to selected activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .126
Unit 6: Analysing your research results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .133
Unit overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .133
Learning outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .133
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .134
Exploring relationships using correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .155
Looking back and looking forward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .164
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .165

Feedback to selected activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .166
Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .171

iv

Practitioner research and evaluation skills training in open and distance learning

Getting and analysing
quantitative data

MODULE

A3

Module overview
You have seen in the earlier modules that there are two broad approaches to
collecting research data: qualitative methods and quantitative methods.This
module looks at the latter type.Through studying the module you will gain an
overview of some of the key issues in collecting and analysing quantitative
data.

You will also learn some of the common methods of statistical analysis of
numerical data, although this is not a module on statistical methods in general
– that is far too large a topic to treat here. Instead, I have concentrated on
showing you how to use some of the basic analytical tools that you can find in
Microsoft Excel.
During the module you will explore and learn about:
䉴 deciding what data you need in order to answer given research questions
䉴 interpreting quantitative data
䉴 types of quantitative data
䉴 methods of summarising quantitative data
䉴 methods of describing patterns and trends in data
䉴 methods of collecting quantitative data, including questionnaire design
䉴 some methods of analysing quantitative data.

Aims of the module
The overall aim of this module is to introduce you to the concepts and
techniques of quantitative research methods in the context of open and
distance learning.
A second aim is to demonstrate that facts involving quantitative data are
‘socially constructed’ and to examine the underlying processes involved.

Our third aim is to enable you to analyse other people’s data and to
appreciate how and why secondary analysis of external quantitative data

Commonwealth of Learning

1

Module A3 – Getting and analysing quantitative data

needs to be done with care if you are to extract meaningful information from
such data.
Our fourth aim is to enable you to analyse quantitative institutional data (e.g.
data on students, their courses and their marks) that already exist in order to
extract meaning and information from that data, using methods such as
averages, measures of spread and trend analysis.
Our final aim is to help you develop the skills of doing institutional research
from scratch.You will look at how to decide what data to collect, which
methods to use and how to design your data collection instruments so that
they will yield valid and reliable results.

Module objectives
When you have worked through this module, you should be able to:
1 Identify the sorts of research questions that can be answered by a
quantitative approach.
2 Calculate percentages as a means of comparing data.
3 Calculate averages as a means of summarising data.
4 Calculate some common measures of dispersion for data.
5 Explain the ideas of validity and reliability and identify methods of
maximising these in your research.
6 Design effective instruments for collecting quantitative data.

Module organisation
The module is structured is in seven parts: this introduction and six units, as
follows.
This introduction: (1 hr)
Unit 1: Introduction (2 hrs)
Unit 2: What do we mean by quantitative data? (1 hr)
Unit 3: Analysing other people’s data (10 hrs)
Unit 4: Quantitative institutional data (9 hrs)
Unit 5: Doing institutional data ‘from scratch’ (9 hrs)

Unit 6: Analysing your research results (9 hrs)
Each unit is made up of the following components:
䉴 an introductory paragraph or two that provide an overview of the unit, its
focus and outcomes

2

Practitioner research and evaluation skills training in open and distance learning

Overview

䉴 a range of activities for you to engage in, many based on Excel
spreadsheets
䉴 unit summary
䉴 feedback on your responses to the questions or problems posed in each
activity.

Requirements
In order to make your learning more interactive, and hence more efficient, we
have included lots of practical exercises.These use numerical data and we take
you through all of the necessary calculations.
We do not assume any great mathematical knowledge. If you understand the
concepts of addition, subtraction, multiplication and division then you should
be able to manage.
However the activities will use the general piece of office software called
Microsoft Excel.This is normally built into desktop computers as standard
nowadays. It will be possible to work through the module without Excel, but
you are strongly advised to get access to it.

Excel
To carry the activities out exactly as laid down you will need access to Excel
Version 4 or above. However you should be able to open the data in earlier
versions and still carry out the exercises. Some of the instructions might have
to be interpreted and adapted.
If you are fairly experienced in Excel, you will be able to carry out the tasks
relatively quickly.
For novice Excel users we have tried to spell out what you need to do. If you
get stuck you can turn to the Model worksheet in each workbook where we
have carried out all of the stages.The letters in brackets, e.g. (W1) tell you
where to look on the relevant Model sheet.
In the Excel instructions, an arrow ➞ is used as shorthand for click and drag.
So the instruction Edit ➞ Paste means that you click and hold on the menu
item Edit at the top of the screen, then drag or scroll down to Paste and
then release the mouse button.

Commonwealth of Learning

3

Module A3 – Getting and analysing quantitative data

Resources
The following resources are used in this module:
Resource

Name when referred to in our text

Location

Excel workbook Women

Women

Resources File

Excel workbook M101

M101

Resources File

Excel workbook Summarising

Summarising

Resources File

Excel workbook Analysing 1

Analysing 1

Resources File

Excel workbook Analysing 2

Analysing 2

Resources File

Excel workbook Correlation

Correlation

Resources File

Teaching boxes
We have used two types of ‘teaching boxes’ in the text as follows:
Statistical note:

Provides additional explanations of some of the statistical terms and methods
that I discuss.
Excel note:

Provides additional information and explanation on how to use Excel.

4

Practitioner research and evaluation skills training in open and distance learning

Introduction

U N I T

1
Unit overview
We live in an age where it is claimed that policy decisions are based on facts
rather than hunches, prejudices and rumours.This is commonly referred to as
‘evidence-based decision-making’.The field of education is no exception,
where it is felt that this ‘evidence’ is needed both to decide between policy
options and also to evaluate the success or failure of past policies.
Furthermore, this ‘evidence’ tends to be ‘facts and figures’ or ‘statistics’. I think it
is fairly uncontentious to say that managers, bosses, civil servants, politicians
prefer quantitative data to qualitative. Faced with an argumentative audience
they like to be armed with charts, spreadsheets and survey results.
This unit will introduce you to:
䉴 some of the difficulties of deciding just what a fact is
䉴 some of the issues that arise when we try to describe a system or a
situation using statistical data.

Learning outcomes
When you have worked through this unit, you should be able to:
1 Discuss the difficulties of deciding just what a fact is.
2 Explain why all knowledge can be seen to be provisional.
3 Illustrate some of the difficulties of using statistical data to describe a
situation such as course enrolments in an ODL institution.
I want you to begin with a fairly light-hearted activity to get you thinking
about ‘facts and figures’

Activity 1

10 mins

Which of these statements are ‘facts’ and why?
1 Paris is the capital city of France.
2 The author of this module is 21 years old and 2 metres (approximately 6 ft 7 ins) tall.
3 Mount Everest is the world’s tallest mountain at 8848 metres high.

Commonwealth of Learning

5

Module A3 – Getting and analysing quantitative data

4 In 2003 The Sukhothai Thammathirat Open University (STOU) in Thailand, with over
300,000 students, was the biggest university in the world.

The feedback to this activity is at the end of the unit 䉴
One thing that we can conclude from this last activity is that good research
should tell you something about precision levels and how much confidence
you should place in the results.
Statistical note: Accuracy and rounding
Numbers are frequently ‘rounded’ before being made public. For example, the number 7.98
may be ‘rounded up’ to 8 and the number 632 may be rounded down to 600. In the case
of Everest the height has been rounded to the nearest metre. The actual result would have
been somewhere between 8847.50001 and 8848.49999.
The measurement method may have been such that this was the greatest level of accuracy
that could have been claimed. The actual result may have been 8848.234632 and this was
then ‘rounded down’ by the scientists because they knew that the instrument used was not
really that accurate. For example, on a hotter day it may have given a slightly higher
reading. In reality they have probably made lots of measurements and then calculated a
middle or ‘average’ value which they then published as their best estimate. (Statistical
averages will be covered later in this module.)
You will frequently see results being presented in social research that exaggerate the
accuracy of the measurement tool. Do not be impressed if the results say that 10.256% of
students disliked the course when in reality it refers to a survey of 100 students where 19
replied and only 2 gave this answer.

The provisional nature of knowledge
Karl Popper, a 20th century philosopher (but one who repeated and
expanded on the ideas of his predecessors) maintained that a theory could
never be completely verified. Even if it has been rigorously tested over a
long period of time, the most that one could say was that the theory has
received a high measure of corroboration. It can be provisionally retained as
the best available theory until it is finally falsified (if indeed it is ever falsified),
and/or is superseded by a better theory.
In my opinion, a strong case can be made for the view that ‘all knowledge is
provisional’ in the Popperian sense. However, I think it is undeniable that all
knowledge, including ‘facts and figures’, is socially constructed.This has led
people like Henry Ford (the founder of the US car company) to proclaim that
‘there are lies, damn, lies and statistics’. Many people in the general public
share his feelings, namely that figures can be manipulated to prove anything
and that they should not be trusted.
I do not take such a negative, cynical view. My position is more a sceptical,
questioning one:

6

Practitioner research and evaluation skills training in open and distance learning

Unit 1: Introduction

䉴 while all knowledge is provisional, some forms of knowledge can be relied
upon more than others
䉴 the trust one places in a piece of knowledge depends to a great extent
upon whether appropriate research methods have been used and how
clearly these have been documented
䉴 arguments in favour of quantitative methods rather than qualitative
methods, or vice versa, will never be conclusive
䉴 in general, the ‘truth’ about a social situation is best approached by a
combination of methods, both quantitative and qualitative
䉴 there are certain rules for constructing and presenting knowledge
䉴 one must be cautious when interpreting knowledge.
Now let’s move on to learning the rules of quantitative methods and how to
apply them.

The rules of quantitative methods and how to
apply them: an introductory case study
Rather than beginning with a long list of the ‘whys?, ‘hows?’ and ‘whens?’ of
quantitative research, I am going to begin with a case study in open and
distance learning that I hope you will find realistic. I thought that this would be
more interesting for you, and more like the reality of practitioner research. At
this stage I will not go into the details of the statistical techniques that I am
using.The important things at the moment are the processes involved in doing
this type of research.

Case studies
The term ‘case study’ is used in many areas and has several meanings. For
example, it can mean the selection and in-depth study of a single ‘case’ –
perhaps one particular ODL institution. By looking intensely at one, complex
example it is intended to give an insight into the context of a problem as well
as illustrating the main point.
Here we are using ‘case study’ in the sense of a student-centred activity based
on a topic that demonstrates theoretical concepts in an applied setting.
In this case study we want you to imagine that you are Abida Quuyaam. As we
explained in Module 1 and the User Guide, Abida is a researcher at Auranzeb
Open University (AOU) and has been working there since she completed her
degree in sociology six years ago. She is part of the Evaluation and Research
Group (ERG) at the university where she works as a junior researcher at the
unit. Besides herself, there is another junior researcher, a senior researcher and
a director.Their mandate is everything and anything the Vice-Chancellor
deems necessary to be investigated, from compiling statistics for different

Commonwealth of Learning

7

Module A3 – Getting and analysing quantitative data

government departments, evaluations of programmes and research projects
that come from abroad.
The Vice-chancellor wants to present findings to the Minister of Education to
show how successful AOU has been in enrolling students on its teacher
training programme. He gives her the data on enrolments on the programme
for the last three semesters and asks the ERG to produce a graph to illustrate
the rise in numbers (Table 1).
Table 1 Enrolments on the teacher training programme at AOU
(Semesters 8 to 10)

Number of enrolments

Semester 8

Semester 9

Semester 10

94

197

320

Abida goes ahead and constructs the graph shown in Figure 1and submits it to
the Vice-chancellor. He, being an astute person, suggests that it be re-drawn as
in Figure 2 because he feels that this shows the growth in numbers better.The
second graph is then sent off to the Ministry.
Figure 1 Enrolments on the teacher training programme at AOU
(Semesters 8–10)
350

300

250

200

150

100

50

0
Semester8

8

Semester9

Semester10

Practitioner research and evaluation skills training in open and distance learning

Unit 1: Introduction

Figure 2 Enrolments on the teacher training programme at AOU
(Semesters 8-10)
350

300

250

200

150

100

50

0
Semester 8

Activity 2

Semester 9

Semester 10

10 mins

Imagine that you are the Minister of Education.
1 What would you make of Abida’s graph (Figure 2)?
2 Would you be impressed by AOU’s performance in teacher training?

The feedback to this activity is at the end of the unit 䉴
Statistical note: Graphs
Graphs (or charts as they tend to be called in Excel) are important and powerful tools. The
horizontal line is called the x-axis and tends to be used for the categories being looked at
such as age, gender, semester, etc. The vertical line is called the y-axis and tends to be used
for the numbers in each category.
The plural of axis is axes.
There are generally accepted rules for the shape of graphs. The default shape produced by
Excel is usually in line with these rules.

Commonwealth of Learning

9

Module A3 – Getting and analysing quantitative data

The role of the researcher in presenting data
Now, what about Abida? Did she behave appropriately?
Well as a junior researcher she is probably not in a very powerful position.To
some extent she has to do exactly what she is told. However, I believe that it
is part of Abida’s professional role to raise questions. She might only be able
to raise them with her line manager, who might block them or produce
convincing counter-arguments. On the other hand her ideas might go further
up the chain and actually effect what goes to the Minister.
Let’s proceed by suggesting some of the questions Abida might have asked.
What should have been her concerns?
Just like the Minister, she should have been concerned about constructing a
graph based on just three data points.
Let’s imagine that in this case she was able to access the information for the
previous eight semesters and that the full data set is in Table 2.
Table 2 Enrolments on the teacher training programme at AOU
(Semesters 1 to 10)
Semester

1

2

3

4

5

6

7

8

9

10

Enrolments

97

122

285

133

137

144

122

94

197

320

The numbers from Table 2 are plotted in Figure 3.
Figure 3 Enrolments on the teacher training programme at AOU
(Semesters 1-10)
350

300

250

200

150

100

50

0
1

2

3

4

5

6

7

8

9

10

Semester

Activity 3

3 mins

What does Figure 3 tell you about the growth of enrolments over the ten semesters?

The feedback to this activity is at the end of the unit 䉴

10

Practitioner research and evaluation skills training in open and distance learning

Unit 1: Introduction

In order to discover underlying trends Abida could use statistical techniques to
smooth out this graph.These tools attempt to remove random fluctuations in
the data.This topic is covered in more detail later in this module. Here we just
illustrate the concept in Figure 4 where we have added the ‘linear trend line’ –
that is the straight line that best represents all of the data mathematically.
Figure 4 Enrolments on the teacher training programme at AOU
with trend line added (Semesters 1–10)
350

300

250

Linear trend
200

150

100

50

0

Activity 4

3 mins

What does the trend line in Figure 4 tell you about the growth of enrolments over the ten
semesters?

The feedback to this activity is at the end of the unit 䉴

Other factors behind the enrolment pattern
Abida might also have the time, the intellectual curiosity and the necessary
data, to investigate other factors that lie behind the enrolment patterns.
Let’s say that the programme consisted of three courses. Course A
‘Curriculum Design’ run in semesters 1, 2, 3, 9 and 10. Course B
‘Developmental Psychology’ run through semesters 3 to 10. Course C
‘Teaching Methods’ only started in semester 10.The enrolments for each
course in each semester are shown in Table 3.

Commonwealth of Learning

11

Module A3 – Getting and analysing quantitative data

Table 3 Enrolments on the teacher training programme at AOU
(Semesters 1 to 10)
Semester
Course A

1

2

3

97

122

127

Course B

4

158

5

133

137

6

144

7

122

8

94

9

10

97

98

100

112

Course C

110
Total

97

122

285

133

137

144

122

94

197

320

Average
enrolments per
course

97

122

143

133

137

144

122

94

99

107

Now, if we graph these numbers (Figure 5) we see that each course is
different.
Figure 5 Enrolments on the teacher training programme at AOU
(Semesters 1–10)
180

160

140

120

100

Course A
Course B
Course C

80

60

40

20

0
1

2

3

4

5

6

7

8

9

10

Semester

Activity 5

5 mins

Describe the different enrolment patterns of three courses, as shown in Figure 5.

The feedback to this activity is at the end of the unit 䉴
Looked at in this way we can see that the peak in enrolments in Semester 3
was because Course B started then with a lot of students.The peak in
Semester 10 occurred because it was the only semester in which all three
courses were running. In fact if you calculate the average number of
enrolments per course (these number are shown in Table 3) then you can see
that there were six semesters that had a higher number of enrolments per
course than Semester 10. Growth at AOU does not seem to be due to

12

Practitioner research and evaluation skills training in open and distance learning

Unit 1: Introduction

greater student demand for each course, but merely the provision of more
courses.

Summary
In the case study we have been looking at quantitative data. We have not
infringed any methodological rules (except, perhaps, when we ‘squeezed’
Figure 2) but we have come up with some different stories and explanations.
This shows that little if anything is self-evident from quantitative data. It has to
be assembled, arranged, presented and interpreted by people – or ‘socially
constructed’.
Secondly, how it is done can depend a great deal on the skills and interests of
the researcher.
Thirdly, the context of the data is important. Abida has shown that by adding
in data from more semesters and breaking it down into courses, we gain a
richer understanding and a more sceptical view of enrolment growth.
However, if the Vice-chancellor knows that there are more courses coming on
stream, and that they seem to attract about one hundred students each, then
his optimism may well be justified.
Finally, there are clearly issues of power involved when it comes to decisions
about what questions are asked, what data is collected, and how it is analysed,
presented and interpreted.

Feedback to selected activities

Feedback to Activity 1
1 Paris is the capital city of France.

This seems like a ‘fact’ to me because that is what my book on France says.
(Pedantically you could say that it is a ‘partial fact’.There is a town in the USA
called Paris that is not the capital of France.) So ‘facts’ don’t necessarily
contain ‘figures’.
2 Alan Woodley is 21 years old and 2 metres (approximately 6
ft 7 ins) tall.

You would only have to meet me to see that this is not true. So the obvious
point is that ‘figures’ do not necessarily constitute ‘facts’. For them to be facts
they have to be correct.

Commonwealth of Learning

13

Module A3 – Getting and analysing quantitative data

3 Mount Everest is the world’s tallest mountain at 8848 metres
high.

Well I looked this up on my computer using GOOGLE, the web-based search
engine. In the first reference that I went to, it said that Everest was 8848
metres high, so I was fairly sure that this was a ‘fact’, but:
䉴 it is not a very precise ‘fact’. All we know here is that, to the nearest metre,
the height is 8848.You could measure a pile of books with your ruler
more precisely than that! However, this apparent lack of precision does not
necessarily mean bad research
䉴 the scientists may have much more precise results but have chosen to
present them in a simplified fashion
䉴 they should not be presenting results to several decimal places if the
measurement techniques do not justify it.
Strictly speaking, even if this ‘fact’ was accurate to several decimal places, it
should have a date attached to it! Apparently Mount Everest is very slowly
getting taller at the rate of two inches (5 centimetres) per year as the
geological movements that created the mountain range continue to force it
upwards.
I have trusted what I have found on my computer but people can put up any
‘facts’ they want to on a website.You need to know whether the source is
reliable, or you should check several sources. In this instance I went to a
second website and it said that the latest estimate using satellite technology
was 8872 metres.
A case could be made that the tallest mountain isn’t Everest but Hawaii’s
Mauna Kea, which rises to a height of 9500 metres from the seabed. It just
happens that most of it is under water and mountains are traditionally
measured from sea level. (Of course, this also raises the question of ‘sea-level’
– how do we measure it and is it a constant?) A third contender is
Chimborazo in Ecuador. Because of the ‘equatorial bulge’ its peak is actually
the furthest from the centre of the earth. My point is that you need to know
the assumptions and definitions that lie behind the ‘facts’.
There are also cultural aspects to this ‘fact’. When asked to name the tallest
mountain you may get different answers depending upon where you are. In
Tibet the local name for the mountain that most of us know as Everest is
Chomolungma. In Nepal it is called Sagarmatha. Early British surveyors labelled
it Peak XV and in 1856 Surveyor General Andrew Waugh, unaware of local
names, named the mountain after his predecessor, George Everest.
4 In 2003 The Sukhothai Thammathirat Open University
(STOU) in Thailand, with over 300,000 students, was the
biggest university in the world

I think that this is probably a ‘fact’. I looked this up on the ICDL database,
which is a reputable source that I trust (http://www-icdl.open.ac.uk/).

14

Practitioner research and evaluation skills training in open and distance learning

Unit 1: Introduction

However, given the variety of ways that students are counted in different
institutions, I would like to know what definition of ‘student’ they were using.
For example, if a person is registered simultaneously on four courses, do they
count as one or four students? Is this the number of students studying at one
point in time, or the number who studied over a given time period such as a
year?
To compare STOU with other universities one needs a standard form of
measurement. One such system is full-time equivalent students (FTE’s). Many
of STOU’s students are studying part-time so it can be argued that they
should only count as a fraction of a student.This fraction would depend upon
what proportion of a full-time load they were carrying.
I would also want to check out which other universities had been included
and how. For example, if the Chinese Central Radio and Television University
and its Regional Television Universities were considered to be one university
then it might be bigger than STOU.Then there is the growing number of
virtual universities such as the University of Phoenix that would need to be
looked at.

Feedback to Activity 2
Well, as a busy Minister, I would be pleased to receive the information in the
form of a graph. Most people find them easier to digest than tables of figures.
They enable you to see patterns and trends in the data.
However, I don’t think that I would be convinced by a graph that only has
three points on it. I would not be confident that you could extend that line
into the future to predict a similar rate of growth.
I would also be suspicious of the shape of the graph. It looks as though the
lower axis has been shortened to make the line on the graph steeper.
I would like to see some comparative figures from other institutions.
Even if I was impressed by the growth in enrolments, I would like to see some
figures on student progress as well.

Feedback to Activity 3
You can see immediately that the graph does not show a pattern of
continuous growth over the ten semesters. In fact, there was also a peak in
the third semester that was almost as big as that in the tenth semester.

Feedback to Activity 4
This trend line still indicates a general increase in enrolments, but a much
smaller one than suggested by just the three most recent semesters.

Commonwealth of Learning

15

Module A3 – Getting and analysing quantitative data

Feedback to Activity 5
The enrolments for Course A increased over the first 3 semesters but were
quite low when it resumed in Semester 9. Course B showed a general decline
over time and Course C had only got data for one semester, the most recent
one.

16

Practitioner research and evaluation skills training in open and distance learning

What do we mean by
quantitative methods?

U N I T

2

Unit overview
This unit introduces you to the idea of quantitative research methods by
considering the types of questions such methods could be used to answer.

Learning outcomes
When you have worked through this unit, you should be able to:
1 Describe the basic difference between quantitative and qualitative research
in terms of their outcomes.
2 List the types of questions that are suited to the quantitative approach.

Introduction
As you have seen earlier in this series, the stages of a research project are
typically:
䉴 Design the research question.
䉴 Identify the population to be studied.
䉴 Select the research tools for data collection.
䉴 Collect the research data.
䉴 Analyse the research data.
䉴 Interpret the results.
䉴 Present the results.
These stages are the same, whatever the size of the project; and regardless of
whether the project is essentially qualitative or quantitative.
The essential difference between the qualitative and quantitative approaches is
in their outputs. Put at its simplest, quantitative research is about measuring
things in a way that can give meaningful numerical results. It is what
researchers in the physical sciences do all of the time. Qualitative research
aims for a subjective understanding of a situation using non-numerical
results.

Commonwealth of Learning

17

Module A3 – Getting and analysing quantitative data

However, while just about any research question in ODL could be formulated
in a way that could be answered by using quantitative or qualitative methods,
there are certain types of question that lend themselves more to a
quantitative approach.

Which questions can be answered with a
quantitative approach?
As we outlined in Module 1, the range of topics and areas of inquiry that can
involve institutional or practitioner research is huge. It is also the case that just
about any research question in ODL could be formulated in a way that could
be answered by using quantitative or qualitative approaches, or a combination
of the two. However, there are certain types of question that lend themselves
more to a quantitative approach and there are certainly situations where
numerical answers are expected and are more appropriate.
To give you some idea of the types of question that quantitative methods
can be used to answer, we list some examples below.To keep things simple,
they all relate to the subject of student age. For each one, we will describe
how a quantitative researcher might set about the task and we will introduce
some of the technical language involved. (These are the words in bold. Don’t
worry if you do not understand some of them.Their meaning will become
clear later in the module.)

Descriptive questions
Example

How old are our students this year?

Purpose

To provide descriptive data.

Source

Probably our institutional database.

Pre-collection issues

We will need to establish working definitions. For example, do we mean all
students registered at a particular date. Is it age on January 1st?

Post-collection issues

We will probably need to group the data into age bands.
We will use descriptive statistics such as frequencies.
The data will be tabulated in order to condense and summarise the information.
We will probably draw a graph or chart based on raw numbers or
percentages.
We may summarise the age distribution using a measure of central tendency
and a measure of dispersion.

18

Practitioner research and evaluation skills training in open and distance learning

Unit 2: What do we mean by quantitative methods?

Comparative questions
Example

How is this age distribution different from other institutions like us?

Purpose

To compare. e.g. in this case to compare a number of different student
populations.

Source

Comparative data may be gained from the government publications, from
published research, from direct collaboration with other institutions both inside and
outside one’s own country.

Pre-collection issues

We will need to decide which institutions are appropriate for comparisons.

Post-collection issues

Further data manipulation may be necessary if, for example, other institutions have
used different age bands.
If necessary, any differences that are found between age distributions in the different
institutions can be tested for statistical significance.

Trend questions
Example

Is the age distribution of our students changing over time?

Purpose

To identify the long-term direction in which the data is moving, e.g. is the average age
growing? declining?

Source

Historical data will be needed to be extracted from the institutional database.

Pre-collection issues

Which should be our base year?

Post-collection issues

This question requires trend analysis.
Regression or other curve-fitting techniques will be used.

Relationship questions
Example

Do young people perform as well as older students?

Purpose

To find out whether one factor (e.g. performance) seems to be linked to another factor
(e.g. age).
It is important to note that, even when we can show that a link exists, that does not
necessarily mean that there is a causal relationship between the two factors.

Source

Additional institutional data will be required on student performance.This might be
whether the student dropped out or not (a nominal scale variable), what position in
the class they came (an ordinal scale variable) or what exam score they gained (an
interval scale variable). (Age is a ratio scale variable.)

Pre-collection issues

Do we need to control for other variables such as previous educational
qualifications which may disguise the true relationship?

Post-collection issues Depending upon the type of performance variable used, an appropriate correlation
or contingency table technique would be selected.

Commonwealth of Learning

19

Module A3 – Getting and analysing quantitative data

Explaining questions
Example

Why do so many young people drop out of our courses?

Purpose

To look for the reasons for an effect that we have already observed, e.g. differential
drop-out rates.

Source

A postal survey using a self-completion questionnaire might be made of young
students who had dropped out, asking them for their reasons.

Pre-collection issues

A random or stratified sample of young students who had dropped out would
be drawn from the institutional database.
A control group of older students who had dropped out might also be used for
comparison purposes.

Post-collection issues

The data would be coded, edited then keyed into a computer and cleaned.
The data would be analysed using frequency tables and cross-tabulations.
Statistical analysis of the institutional database might be carried out to help answer this
question.Various forms of multi-variate analysis could see whether this is true
across the curriculum and whether age appears to be the causal factor.

Attitude questions
Example

Do young people like studying our courses?

Purpose

To find out how people feel about a particular issue.

Source

A course feedback survey across the age range could be carried out as described
in the previous question.

Pre-collection issues

What is meant by ‘like’? We need to operationalize our terms so that people can
answer in numerical terms.

Post-collection issues Crosstabulations and correlational techniques could be used to see whether
there is a relationship between age and attitudes to some or all of the courses.

Predictive questions
Example

What will happen if we become more attractive to young people?

Purpose

What are the implications for the institution and its various subsystems if this happens?

Source

The institutional database.

Pre-collection issues

None.

Post-collection issues Statistical modelling techniques can be used to predict effects on course
numbers in different areas, drop-out rates, the demand on financial assistance funds, etc

20

Practitioner research and evaluation skills training in open and distance learning

Unit 2: What do we mean by quantitative methods?

Summary
In this unit, you have explored some of the types of question that can be
answered using quantitative techniques.

The rest of the module
This is the end of the introductory part of the module.The rest of the
module has been structured in a particular way that we hope will improve
your learning and retain your interest.
It is in three broad sections:
1 The first is about you carrying out secondary analysis of external data.
2 The second chiefly concerns institutional data that is based on information
collected on a regular basis; the third is where you collect data for a
specific research purpose.
3 The third and last section will help you to plan your own study in which
you wish to collect quantitative data.
We introduce particular research methods and statistical techniques as we go
along whenever they become relevant.
The approach is meant to be very interactive.You will be guided through a set
of activities using real data in a spreadsheet software package called Excel. If
you do not have access to Excel, you should be able to carry out the
calculations by hand, or by using a calculator. If you use a statistical software
package such as SPSS, you may wish to use that instead.

Commonwealth of Learning

21

Analysing other peoples data

U N I T

3
Unit overview
This unit is designed to introduce you to:
䉴 the basics of using Excel
䉴 some simple methods of analysing student data using Excel.

Learning outcomes
When you have worked through this unit, you should be able to:
1 Calculate percentages with and without Excel.
2 Calculate totals using Excel.
3 Copy formulae in Excel by dragging.
4 Copy and paste values in Excel.
5 Copy and paste formats in Excel.
6 Sort data in Excel.
7 Apply these methods to a case study on enrolments and exam passes.

Introduction
While most books on research methods