THE BASICS: EVIDENCE-BASED PRACTICE FOR PHYSICAL THERAPISTS AND PHYSICAL THERAPIST ASSISTANTS, ONLINE TRAINING MODULE

THE BASICS: EVIDENCEBASED PRACTICE FOR
PHYSICAL THERAPISTS AND
PHYSICAL THERAPIST
ASSISTANTS,
ONLINE TRAINING MODULE
1

Jessica Lambeth, MPT
jessicamlambeth@yahoo.com

Module Purpose
The purpose of this online training module is
to share the basics of evidence-based practice
(EBP). This module focuses on general concepts
of EBP, with clinical scenarios related to schoolbased physical therapy.

2

Module Purpose, continued
The main emphasis of this module is not to
promote certain treatments or tests and measures

or “tell you” what the recent literature reveals about
pediatric physical therapy practices. As will be
discussed in the coming slides, the “research
answer” alone is not the correct answer. In
addition, EBP emphasizes the necessity of learning
to perform searches and evaluate the material
independently, related to clinicians’ and patients’
current circumstances. While initially time
consuming, the results are worthwhile.
3

Module Purpose, continued
For those without a recent EBP background
or training, this module will not result in immediate
efficiency in literature searches, statistical
interpretation, etc. Hopefully it will, however,
encourage more questions and motivation for EPB.
It will also enable the NC DPI to focus educational
sessions on meeting needs in the area of EBP and
to gauge the current knowledge of school-based

PTs and PTAs. Your feedback and comments will
help to plan future learning opportunities and
resources.
4

Pretest
Please stop here and complete the pretest if
you have not already done so. Please also keep
track of your time, as directed in the instructions.
Also, “I don’t know” answers have been
included to ensure that we receive appropriate
feedback. If you really do not know the answer, in
the pretest or the posttest, please indicate that in
your response. This will greatly assist in providing
necessary learning opportunities in the future, as
well as help us to evaluate the effectiveness of the
online module format and material.
5

References:

The content of this EBP module is largely based
on curriculum from courses within the transitional
Doctor of Physical Therapy program at Rocky
Mountain University of Health Professions in Provo,
Utah. (web address: www.rmuohp.edu)

6

References, continued:
Much of the material within this online training module can be
found in the following texts:
Guyatt G, Rennie D. Users' Guides to the Medical Literature- Essentials of
Evidence-Based Clinical Practice. Chicago: AMA Press; 2002.
Jaeschke R, Singer J, Guyatt GH. Measurement of health status:
ascertaining the minimal clinically important difference. Control Clin Trials
1989;10:407-15.
Portney LG, Watkins MP. Foundations of Clinical Research: Applications to
Practice. 2nd ed. Upper Saddle River, NJ: Prentice-Hall Inc, 2000.
Rothstein JM. Autonomous Practice or Autonomous Ignorance? (Editorial)
Physical Therapy 81(10), October 2001.

7

References, continued:
Much of the material within this online training module can be
found in the following texts:
Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-based
Medicine: How to Practice and Teach EBM. 3rd ed. Edinburgh: Churchill
Livingstone; 2005.
All clipart came from Microsoft Office:
http://office.microsoft.com/en-us/clipart/default.aspx.
Accessed 05/09/2009.

8

Module Outline
1) What is EBP? Slides 9-22
2) Statistics Review: Basic Research, Slides 23-115
3) How to Search for EBP, Slides 116-152
4) How to Interpret Research Related to EBP, Slides
153-174

9

Section 1 Outline
What is Evidence-Based Practice?
• Evidence-Based Practice: General Introduction,
10-12
• Survey Results (NC school-based PTs view of
EBP),13-18
• Guiding Steps to Practice EBP,19-20
• Two Fundamental Principles of EBP, 21
• Best Research Evidence, 22

10

Evidence-Based Practice:
General Introduction
EBP is the integration of the best research evidence,
clinical expertise, and the patient’s values and
circumstances

• Best Research Evidence: valid and clinically
relevant research with a focus on patient-centered
clinical research
• Clinical Expertise: use of clinical skills and
experiences
• Patient’s Values and Circumstances: the
patient’s unique preferences, concerns, and
expectations in his or her setting
11

(Straus et al, 2005)

Evidence-Based Practice:
General Introduction
EBP is Not:
• Focused only on research studies
• Only to be used or understood by professionals
who routinely participate in research studies
• A discouragement from trying new treatment
 There may be little or no research on a

particular topic, or studies with small sample
sizes may have lacked the power to
demonstrate statistical significance (as later
explained in the statistics section)
12

Evidence-Based Practice:
General Introduction
“Because RCTs are so difficult, we will always have
areas that lack evidence, we will need to find other
credible research approaches to supply evidence.
Keep in mind that an absence of evidence is
different from negative evidence. An absence of
evidence is not an excuse to ignore the growing
body of data available to guide practice.”
(RCT = randomized clinical or controlled trials)

13

(Rothstein 2001)

Survey Results
Survey responses from North Carolina SchoolBased PTs/PTAs about EBP:
• Survey distributed at the NC Exceptional
Children’s Conference, Nov 2008
• 41 Respondents (
www.med.unc.edu/ahs/physical/schoolbasedpt for
detailed results)

• 73% had participated in a conference or course
on the use of EBP in the last 5 years
• Of those that participated in a course,
respondent data suggests the course changed
their view of EBP, but their use and practice
14
of EBP was changed to a lesser degree.

Survey Results, Continued…
• Highest frequency response as to why we should
use EBP  positive impact on our clinical practice

• 39 of 41 respondents agreed that EBP is relevant
and necessary for PTs in the school system (2 left
that question blank)
 Highest frequency responses as to why it is
relevant and necessary: A focus on EBP results in
improved clinical practice and provides validation
and justification for our role as school-based
PTs/PTAs
15

 

Survey Results, Continued…
• When respondents were asked if they were
comfortable searching for and using EBP:
• Yes: 17 (41%),
• No: 18 (44%),
• No Response: 6 (15%)
• Internet was the most frequently
used source for searching EBP,

but continuing education courses
ranked highest for preference.
16

Survey Results, Continued…
• A majority of the respondents were comfortable:
 Determining the level of evidence and
interpreting the authors’ conclusions
• A majority of the respondents were not comfortable:
 Interpreting statistics, even though statistical
knowledge is often helpful to evaluate conclusions
drawn by the author

17

Survey Results, Continued…
The primary barriers to searching and using EBP, as
listed by NC school PTs/PTAs:
1. Time
2. Access

3. Uncertain how to search for EBP
Factors that enhance the search and use of EBP,
as listed by NC school PTs/PTAs listed:
1. Additional time
2. Education on the appropriate use of EBP
3. Education in how to access EBP resources
18

Survey Results, Continued…
Questions generated from the survey:
• How to efficiently and effectively increase the
knowledge and practice of EBP by NC school PTs/
PTAs?
• How to address barriers to using and searching
for EBP?
• How to enhance use and access of EBP in
school-based practice?

19

Guiding Steps to Practice EBP
1.Analyze what we know and what we do not know, in
relation to improving our clinical practice. Form
answerable questions to address any gaps in our
knowledge.
2.Search for and find the best research evidence to
address our questions.
3.Critically appraise the information, based on its
validity, impact or size of effect, and applicability.
20

(Straus et al, 2005)

Guiding Steps to Practice EBP
4.Integrate information gathered from the best
research evidence with clinical expertise and the
patient’s values and circumstances
5.Evaluate the effectiveness of any intervention taken
based on steps 1-4, and the effectiveness and
efficiency of the process

21

(Straus et al, 2005)

Two Fundamental Principles of EBP
1.“Evidence alone is never sufficient to make a
clinical decision” (page 8)
Consider risks and benefits, costs, inconvenience,
alternative treatment strategies, patient
preferences/values and circumstances.

2.“EBM posits a hierarchy of evidence to guide
clinical decision making” (page 8)
Not all research is equal in terms of relevance and
statistical support, however, that does not mean
lower level evidence is not worthwhile.
(Guyatt

and Rennie use the term Evidence-Based
22
Medicine, EBM)
(Guyatt and Rennie, 2002)

Best Research Evidence
• The three sections that follow will focus on the
“best research evidence” branch of Straus’ three
components of EBM. (Straus et al, 2005)
• Best research evidence is not more important than
the other two branches; it is prominent in this module
because knowledge concerning clinical expertise and
patient values and expectations will vary from
situation to situation.
• Section 2 (Statistics Review, Basic Research) will
provide the background information necessary to
perform effective searches and interpret the best
23
research evidence (Sections 3 and 4).

Section 2 Outline
Statistics Review, Basic Research
•
•
•
•

Types of Research, 25-33
Hierarchy of Evidence, 34-35
Variables, 36-44
Measurement Validity
• Types, 45-48
• Statistics, 49-67
• Sensitivity and Specificity
• Positive and Negative Predictive Value
• Positive and Negative Likelihood
• Receiver Operating Characteristic
(ROC) Curve
• Responsiveness to Change
• Effect Size versus MCID

24

Section 2 Outline, continued…
Statistics Review, Basic Research, continued
•
•

•

Measurement Reliability, 68-72
Descriptive Statistics,73-83
• Frequency and Shape of Distribution
• Central Tendency Measures
• Measures of Variability
Inferential Statistics, 84-115
• Probability
• Sampling Error
• Confidence Intervals
• Hypothesis Testing
• Power
25

Types of Research
Nonexperimental (Observational)
• Descriptive or Exploratory
• No control or manipulation of variables
• Examines populations and relationships
Experimental
• Researcher manipulates/controls variable(s)
• Comparison of interventions or groups,
examines cause and effect
26

(Portney and Watkins, 2000)

Types of Research
Portney and Watkins suggest viewing various designs
as a continuum of research with a descriptive,
exploratory, or experimental purpose.
Certain designs may include elements of more than
one classification.

27

(Portney and Watkins,2000)

Descriptive Research
Examples:
• Case Study: Description of one or more patients,
may document unusual conditions or response to
intervention
• Developmental Research: Examines patterns of
growth and change, or documents natural history
of a disease or condition
• Normative Research: Establishes typical values
for specific variables

28

(Portney and Watkins, 2000)

Descriptive Research, continued…
Examples:
• Qualitative Research: Collection of subjective,
narrative information (rather than quantitative,
numerical data) in an effort to better understand
experiences
• Evaluation Research: Assessment of a program
or policy, often by the collection of descriptive
information through surveys or questionnaires

29

(Portney and Watkins, 2000)

Exploratory Research
Examples:
• Correlational Methods: Examines relationships
between areas of interest, may be used to predict
or suggest, but cannot offer cause and effect
• Cohort and Case-Control Studies: Used often in
epidemiological research to describe and predict
risks for certain conditions
• Methodological Studies: Used to evaluate the
validity and reliability of measuring instruments
• Historical Research: Use of archives or other
records to reconstruct the past to generate
questions or suggest relationships of historical
30
interest
(Portney and Watkins, 2000)

Experimental Research
Example
• Randomized Clinical or Controlled Trial (RCT):
In general, a clinical treatment, or experimental
condition, is compared to a control condition, often
a placebo but in some cases an alternative
treatment, where subjects are randomly assigned
to a group.

31

(Portney and Watkins, 2000)

Experimental Research, continued…
Examples:
• Single-Subject Design: Variation of RCT, study
of an individual over time with repeated
measurement and determined design phases
(Portney and Watkins, 2000)

In an N=1 RCT, a single individual receives
alternating treatment and placebo or alternative
treatment, with the patient and the assessor
blinded to intervention allocation. Objective or
subjective measures are then recorded during the
allocation periods. (Guyatt and Rennie, 2002)
32

Experimental Research, continued…
Examples:
• Sequential Clinical Trial: Variation of RCT,
technique that allows for the continuous analysis of
data as it becomes available, does not require a
fixed sample
• Quasi-Experimental Research: Comparative
research in which subjects cannot be randomly
assigned to a group, or control groups cannot be
used. Lower level of evidence than RCTs.
33

(Portney and Watkins 2000)

Experimental Research, continued
Examples:
• Systematic Review: Combination of several
studies with the same or similar variables, in which
the studies are summarized and analyzed (Guyatt
and Rennie, 2002)

• Meta-analysis: Statistical combination of the data
from several studies with the same or similar
variables, to determine an overall outcome (Portney
and Watkins, 2000; Guyatt and Rennie, 2002)

34

Hierarchy of Evidence for Treatment
Decisions:
Greatest (Top) to Least (Bottom)
1. N of 1 randomized controlled trial
2. Systematic review of randomized trials*
3. Single randomized trial
4. Systematic review of observational studies
addressing patient-important outcomes
5. Single observational study addressing patientimportant outcomes
6. Physiological studies (studies of blood pressure,
cardiac output, exercise capacity, bone density, and
so forth)
35
7. Unsystematic clinical observations
*A meta-analysis is often considered higher than a

Hierarchy of Evidence
Ideally, evidence from individual studies would
be compiled or synthesized into systematic reviews,
with that information succinctly consolidated into
easily and quickly read synopses. All relevant
information would be integrated and linked to a
specific patient’s circumstance. The medical search
literature is still far from this, but working towards that
goal. Efforts include clinical prediction guidelines and
APTA’s emphasis on EBP.
36

(Straus et al, 2005)

Variables
Variables: Characteristic that can be manipulated or
observed
• Types of Variables
• Independent or Dependent
• Measurement Scales/Levels
Classification is useful for communication, so that
readers are aware of the author’s hypothesis of what
situation or intervention (independent variable) will
predict or cause a given outcome (dependent
variable)
37

(Portney and Watkins, 2000)

Variables: Independent or Dependent
• Independent Variable: A variable that is
manipulated or controlled by the researcher,
presumed to cause or determine another
(dependent) variable
• Dependent Variable: A response variable that is
assumed to depend on or be caused by another
(independent) variable

38

(Portney and Watkins, 2000)

Variables: Measurement Scales
• Useful to convey information to the reader about
the type of variables observed
• Necessary to determine what statistical analysis
approach should be used to examine relationships
between variables
• From lowest to highest level of measurement,
the scales are nominal, ordinal, interval, and ratio

39

(Portney and Watkins, 2000)

Variables: Measurement Scales
Nominal Scales (Classification Scale)
• Data, with no quantitative value, are organized
into categories
• Categorizes are based on some criterion
• Categories are mutually exclusive and
exhaustive (each piece of data will be assigned to
only one category)
• Only permissible mathematical operation is
counting (such as the number of items within each
category)
• Examples: Gender, Blood Type, Side of
40
Hemiplegic Involvement
(Portney and Watkins, 2000)

Variables: Measurement Scales
Ordinal Scales
• Data are organized into categories, which are
rank-ordered on the basis of a defined
characteristic or property
• Categories exhibit a “greater than-less than”
relationship with each other and intervals between
categories may not be consistent and may not be
known

41

(Portney and Watkins, 2000)

Variables: Measurement Scales
Ordinal Scales, continued
• If categories are labeled with a numerical value,
the number does not represent a quantity, but only
a relative position within a distribution (for
example, manual muscle test grades of 0-5)
• Not appropriate to use arithmetic operations
• Examples: Pain Scales, Reported Sensation,
Military Rank, Amount of Assistance Required
(Independent, Minimal…)
42

(Portney and Watkins, 2000)

Variables: Measurement Scales
Interval Scales
• Data are organized into categories, which are
rank-ordered with known and equal intervals
between units of measurement
• Not related to a true zero
• Data can be added or subtracted, but actual
quantities and ratios cannot be interpreted, due to
lack of a true zero
• Examples: Intelligence testing scores,
temperature in degrees centigrade or Fahrenheit,
calendar years in AD or BC
43

(Portney and Watkins, 2000)

Variables: Measurement Scales
Ratio Scales
• Interval score with an absolute zero point (so
negative numbers are not possible)
• All mathematical and statistical operations are
permissible
• Examples: time, distance, age, weight

44

(Portney and Watkins, 2000)

Variables: Clinical Example
A study investigates how a strengthening
program impacts a child’s ability to independently
walk. In this case, the strengthening program is the
independent variable and the ability to independently
walk is the dependent variable. Amount of
assistance required (if ranked maximal, moderate,
minimal, independently, not based on weight put on a
crutch or other quantitative testing) would be an
example of ordinal data.
Studies often have more than one independent or
dependent variable
45

Measurement Validity
• Measurement Validity examines the “extent to
which an instrument measures what it is intended to
measure” (Portney and Watkins, 2000)
• For example, how accurate
is a test or instrument at
discriminating, evaluating,
or predicting certain items?

46

Measurement Validity
Validity of Diagnostic Tests
• Based on the ability for a test to accurately
determine the presence or absence of a condition
• Compare the test’s results to known results,
such as a gold standard.
• For example, a test determining balance
difficulties likely to result in falls could be compared
against the number of falls an individual actually
experiences within a certain time frame. A clinical
test for a torn ACL could be compared against an
MRI.
47

(Portney and Watkins, 2000)

Measurement Validity: Types
• Face Validity: Examines if an instrument appears to

measure what it is supposed to measure (weakest
form of measurement validity)
• Content Validity: Examines if the items within an
instrument adequately comprise the entire content of
a given domain reported to be measured by the
instrument
• Construct Validity: Examines if an instrument can
measure an abstract concept
48

(Portney and Watkins, 2000)

Measurement Validity: Types
• Criterion-related Validity: Examines if the outcomes
of the instrument can be used as a substitute
measure for an established gold standard test.
 Concurrent Validity: Examination of Criterionrelated Validity, when the instrument being examined
and the gold standard are compared at the same time
 Predictive Validity: Examination of Criterion-related
Validity, when the outcome of the instrument being
examined can be used to predict a future outcome
determined by a gold standard

49

(Portney and Watkins, 2000)

Measurement Validity: Statistics
Ways to Evaluate Usefulness of Clinical Screening or
Diagnostic Tools
• Sensitivity and Specificity
• Positive and Negative Predictive Value
• Positive and Negative Likelihood Ratios
• Receiver Operating Characteristic (ROC) Curve
The above mentioned statistical procedures are often
used when researchers are introducing (and validating)
the test. Hopefully the values from these operations can
be found tool’s testing manual or in articles evaluating the
tool’s validity within certain populations or settings.
50

Measurement Validity: Statistics
Diagnostic Reference Table
Condition
Present
Absent
Test Result
Positive
Negative

True Positive
a

False Positive
b

False Negative
c

True Negative
d

(Guyatt and Rennie, 2002; Portney and Watkins, 2000;
Straus et al, 2005)

51

Measurement Validity: Statistics
Table Labels:
(a) True Positive: The subject matter has the
condition, and the test accurately identifies the
presence of the condition
(d) True Negative: The subject matter does not have
the condition, and the test accurately identifies the
absence of the condition
(b) False Positive: The subject matter does not have
the condition, and the test incorrectly identifies the
presence of the condition
(c) False Negative: The subject matter has the
condition, and the test incorrectly identifies the
52
absence of the condition
(Portney and Watkins, 2000)

Measurement Validity: Statistics
• Positive test result = the test identifies the condition
as being present;
• Negative result = the test identifies the condition as
being absent
(This may or may not be accurate when compared to
the gold standard).
The test’s sensitivity and specificity provide
information about the accuracy of the test.
53

(Portney and Watkins, 2000)

Measurement Validity: Statistics
Sensitivity
• The ability of the test to obtain a positive test when
the condition is present; the ability to detect a true
positive (a)
• a/(a + c) The proportion that test positive out of
those with the condition

54

(Portney and Watkins, 2000)

Measurement Validity: Statistics
Specificity
• The ability of the test to obtain a negative result
when the condition is absent, the ability to detect a
true negative (d)
• d/(b + d) The proportion that test negative out of
those without the condition
Sensitivity and Specificity are often provided in terms
of percents, from 0% to 100% (low to high)
55

(Portney and Watkins, 2000)

Measurement Validity: Statistics
Helpful Hints to remember
Sensitivity and Specificity
• Sensitivity: SnNout: When a test has a high
sensitivity (Sn), a negative result (N), rules out (out)
the diagnosis.
• Specificity: SpPin: When a test has a high
specificity (Sp), a positive result (P), rules in (in) the
diagnosis

56

(Straus et al, 2005)

Measurement Validity: Statistics
Clinical Example
Example:
You’re choosing between two tests that screen
participation in school based on physical abilities.
A positive result means that the student’s physical
abilities impact his or her participation.

57

Measurement Validity: Statistics
Clinical Example
One test has a high sensitivity, but a low specificity.
A high sensitivity means that a negative test will
effectively rule out students whose physical abilities
do not impact participation.
However, with a low specificity, there may be many
false positives, meaning students may test “positive”
for the condition when, in fact, their abilities do not
impact participation.
58

Measurement Validity: Statistics
Clinical Example
Example:
You’re choosing between two tests that evaluate
participation in school based on physical abilities.
A positive result means that the student’s physical
abilities impact his or her participation.

59

Measurement Validity: Statistics
Clinical Example
The other test has a low sensitivity, but a high
specificity. A high specificity means that a positive
result will effectively rule in the condition.
However, with a low sensitivity, there may be many
false negatives, meaning that students may test
“negative” for the condition, when, in fact, their
abilities do impact their participation.
60

Measurement Validity: Statistics
Predictive Values
• Provided in terms of percentages, 0% to 100%,
low to high
• Positive Predictive Value (PV+)
• Probability that a person with a positive test
actually has the condition
• a/(a + b)
• High PV+ desired for screening tools, to
prevent excessive unnecessary future testing
• Negative Predictive Value (PV-)
• Probability that a person with a negative
test does not have the condition
61
• d/(c + d)
(Portney and Watkins, 2000)

Measurement Validity: Statistics
Likelihood Ratios
• Calculated from the Diagnostic Reference Table
• Requires prior calculation of the pretest
probability of the condition in question
• Easy to use when familiar with the concept, but
requires the use of a probability guide chart or a
nomogram (chart that contains pretest probability
and likelihood ratios, with a ruler connecting those
two points to determine posttest probabilities)

62

(Guyatt and Rennie, 2002)

Measurement Validity: Statistics
Likelihood Ratios, continued
• Positive and negative likelihood ratios are
calculated
• Determines the usefulness of a diagnostic test.
If a positive or negative result will change the
posttest probability of having a condition to alter
the clinician and patient’s course of action, it will
be useful. If the likelihood ratios of the test do not
result in a substantial change of knowledge, the
test most likely will not be useful.
63

(Guyatt and Rennie, 2002)

Measurement Validity: Statistics
Receiver Operating Characteristic (ROC) Curve
• Uses sensitivity and specificity information to
find the probability of correctly choosing between
presence or absence of the condition
For example, a test with an area under the ROC
curve of 0.80, would result in the correct
determination of presence or absence of a
condition 80% of the time.

64

(Portney and Watkins, 2000)

Measurement Validity: Statistics
Responsiveness to chance statistics evaluate a
measurement tool’s ability to detect change over time
• For example, will administration of the test pre
and post intervention reflect a change in status, if
one actually occurred?
• Evaluated by examining the change in scores in
a pretest-posttest design, or using effect size

65

(Portney and Watkins, 2000)

Measurement Validity: Statistics
Effect Size
• Effect size (ES) is a measure of the amount of
difference.
For example, experimental group A increased
their score on a coordination measure by an
average of 15 points, while experimental group B
increased their score an average of 8 points. The
ES between groups would be 7 points,
considering the groups were homogeneous at the
start.
66

Measurement Validity: Statistics
Effect Size, continued…
• An effect size index is a converted effect size, a
standardized measure of change, so that change
scores across different outcome measures can be
compared.
• ES is often displayed as a correlation
coefficient, r
Portney and Watkins note that considerations of ES
vary based on the interpreting clinician, but review
Cohen’s suggestions of scores 0.8 as large
(Portney and Watkins, 2000)

Measurement Validity: Statistics
Effect Size versus Minimal Clinically Important
Difference
• In addition to numerical data revealed and
statistical significance, the clinician should also
consider what amount of change is clinically
meaningful, such as, how great a gain in strength
or endurance will result in a change in function?
This is often referred to as the minimal clinically
important difference (MCID).
68

(Jaeschke et al 1989)

Measurement Reliability: Statistics
Reliability examines a measurement’s consistency
and freedom from error
• Can be thought of as reproducibility or
dependability
• Estimate of how observed scores vary from the
actual scores

69

(Portney and Watkins, 2000)

Measurement Reliability: Statistics
Reliability Coefficient
• Ratio of reliability (many different types with
various symbol representation)
• Range between 0.00 to 1.00;
• 0.00 = no reliability;
• 1.00 = perfect reliability
• Reflection of variance, a measure of the
differences among scores within a sample
70

(Portney and Watkins, 2000)

Measurement Reliability: Statistics
Correlation
• Comparison of the degree of association
between two variables or sets of data
• Used as a basis for many reliability coefficients

71

(Portney and Watkins, 2000)

Measurement Reliability: Statistics
Test-Retest Reliability
Examines the consistency of the results of
repeated test administration
• Traditional Analysis
• Pearson product-moment coefficient of
correlation (for interval or ratio data)
• Spearman rho (for ordinal data)
• Current, sometimes considered preferred,
Analysis
• Intraclass correlation coefficient
72

(Portney and Watkins, 2000)

Measurement Reliability: Statistics
Rater Reliability
• Intrarater reliability
• Reliability of data collection from one
individual over two or more trials
• Interrater reliability
• Reliability of data collection between two or
more raters

73

(Portney and Watkins, 2000)

Descriptive Statistics
Descriptive statistics are used to describe sample
characteristics.
• A sample is a subset of a population chosen for
study. Since data often cannot be collected from an
entire population, the data chosen from a selected
sample is intended to be representative or an
estimate of the population data.
• Distribution: Total set of scores (from a sample) for
a particular variable, given the symbol, n
74

(Portney and Watkins, 2000)

Descriptive Statistics
Frequency and Shapes of Distribution
• Frequency distribution: The number of times
each value, from the variable data, occurred
• Drawing a graph of frequency distributions can
result in shapes that characterize the distributions
• Some graphs are asymmetrical, others are
symmetrical
• A symmetrical graph with a bell-shaped
distribution is referred to as a normal
distribution.
• A skewed distribution presents
asymmetrically
75

Descriptive Statistics
Normal Distributions, when graphed according
to frequency, present in the shape of a bell with the
majority of scores falling in the middle and
progressively fewer scores at either end. It has
special properties in statistics.

76

(Portney and Watkins, 2000)

Descriptive Statistics:
Central Tendency Measures
Used to quantitatively summarize a group’s
characteristics.
• Mode: The score that occurs most frequently
• Median: The middle score in numerically
ordered group of data. If there are an even
number of scores, the median is the midpoint
between the two middle scores
• Mean: The sum of a set of scores divided by the
number of scores, n. Often referred to as the
“average” of a data set.
77
(Portney and Watkins, 2000)

Descriptive Statistics:
Measures of Variability
Variability is the dispersion of scores.
Variability is affected (qualified) by five
characteristics:
• Range
• Percentiles
• Variance
• Standard deviation
• Coefficient of variation
78

(Portney and Watkins, 2000)

Descriptive Statistics:
Measures of Variability
Variability, continued…
Range: Difference between the highest and lowest
scores in a distribution
Percentiles: Used to describe a score’s position
within a distribution, distribution data is often divided
into quartiles, or four equal parts
Variance: (Too in-depth to describe the statistical
background for this module purpose). Reflects
variation within a set of scores, in square units.
Symbolized in sample data by s

79

(Portney and Watkins, 2000)

Descriptive Statistics:
Measures of Variability
Variability, continued
Standard Deviation: Representative of the variability
of scores surrounding the mean, in original units of
measurement. Square root of variance.
• The larger the standard deviation, the more
spread out the variable’s scores are around a mean.
• For example, data set (A) 8,9,10,11,12 and the
data set (B) 4,5,10,15,16 both have a mean of 10.
However the standard deviation for set A is 1.58
while the standard deviation of set B is 5.52.
80
2
• Symbolized in sample data by s .

Descriptive Statistics:
Measures of Variability
Variability, continued…
Coefficient of Variation: The ratio of the standard
deviation to the mean.

81

(Portney and Watkins, 2000)

Descriptive Statistics:
Distributions
Normal Versus Skewed (Non-normal) Distribution
• These theoretical shapes of distribution help
determine what statistical formulas or measures
should be used
• The characteristics of normally distributed data are
constant and predictable. For statistical purposes, the
normally distributed curve is often divided into
proportional areas, each equal to one standard
deviation.
• Data should be examined for “goodness-of-fit” to
see if the sample approximates the normal
distribution.
(Portney

82

and Watkins, 2000)

Descriptive Statistics:
Distributions
Normal Distribution Statistics
• 1st standard deviation on either side of the average
contains 34.13% of the data
• total of 68.62% of the data will be between +1
and -1 standard deviation
• Between 1st and 2nd standard deviation contains
13.59% of the data
• total of 95.45% of the data will be between +2
and -2 standard deviations
[(13.59 times 2) + (34.13 times 2)]

83

(Portney and Watkins, 2000)

Descriptive Statistics:
Distributions
Normal Distribution Statistics, continued
Between 2nd and 3rd standard deviation contains
2.14% of the data
• total of 99.73% of the data will be between +3
and -3 standard deviations
[(13.59 times 2) + (34.13 times 2) + (2.14 times
2)]

84

(Portney

and Watkins, 2000)

Inferential Statistics
Estimate population characteristics from sample data
• Used often when testing theories about the effects
of experimental treatments
• Requires that assumptions are made about how
well the sample represents the larger population
• Assumptions are based on the statistical concepts
of probability and sampling error
• It is important the sample be representative of the
population, so that the results of interventions on
samples can be applied to the entire population of
individuals with those characteristics.
85

(Portney

and Watkins, 2000)

Inferential Statistics
Probability
Probability
• The likelihood that an event will occur, given all
the possible outcomes. Used often in prediction.
Given the symbol p
• Probability may range from p = 1 (certain the
event will occur, 100% probability) to p = 0 (certain
that the event will not occur, 0% probability)

86

(Portney and Watkins, 2000)

Inferential Statistics
Probability
Probability, continued…
Reflective of should happen in the long run, not
necessarily what will happen on a given trial.
For example, if a treatment has a 60% chance of
success, then 60% of people will likely be
successfully treated. That does not mean the
treatment will be 60% successful in an individual,
likely it will either be a unsuccessful or successful
for that individual.
87

(Portney and Watkins, 2000)

Inferential Statistics: Probability
Clinical Example
Probability statistics can be applied to the distribution
of scores
Example, for a normally distributed data set:
Average long jump for a certain group of children is
35 inches with a standard deviation of 4 inches.
Suppose you want to know the probability that a child will
jump within one standard deviation (from 31 to 39
inches)? You know that within one standard deviation on
either side of the mean is 68.2%, so that is the probability
that a child will jump within one standard deviation of the
mean.
88

Inferential Statistics: Probability
Clinical Example
Example, continued…
If you wanted to know the probability that a
child will jump more than one standard deviation
above the mean (greater than 39 inches), you can
refer to the data and calculate 15.86%.
Charts and graphs are available to calculate the data
in between the standard deviations.
89

Inferential Statistics:
Sampling Error
Sampling Error
The difference between sample values and
population values
• The lower the sampling error, the greater the
confidence that the sample values are similar to
the population values.
• To estimate sampling error, the standard error
of the mean statistic is used (too complex to
explain statistical basis in this format)
• The larger the sample size, n, the smaller the
standard error of the mean
90
(Portney and Watkins, 2000)

Inferential Statistics
Confidence Intervals
Confidence Interval (CI)
• Range of scores with specific boundaries, or
confidence limits, that should contain the
population mean
• The boundaries of CIs are based on the sample
mean and its standard error
• Degree of confidence is expressed as a
percentage
• Often, researchers use 95% as a boundary,
which is just slightly less than 2 standard
deviations on either side of the mean
91
(Portney and Watkins, 2000)

Inferential Statistics, Confidence Interval
Clinical Example
A physical therapy treatment program resulted in
the ability for 40 children with a certain disorder to walk
an additional 8 independent steps, on average, within a
certain set of parameters.
The 95% CI for this data was ± 2 steps.
Therefore, we can be 95% certain that the population
mean, or average for all children with this disorder, is
between 6 and 10 extra independent steps.
Said another way, if there were an additional 100 children
with the same condition, 95 of them would likely have an
average that was between an additional 6 to 10
92
additional independent steps following the physical
therapy treatment.

Inferential Statistics
Hypothesis Testing
Hypothesis Testing
Used to decide if effects from an intervention are due
to chance or the result of the intervention
Results in one of two decisions: To accept or
reject the null hypothesis

93

(Portney and Watkins, 2000)

Inferential Statistics
Hypothesis Testing
Hypothesis Testing, continued…
Statistical Hypothesis (also known as the null
hypothesis):
Any observed difference (as in pretreatment to posttreatment or treatment compared to a placebo) is due
to chance.
When the null hypothesis is rejected, the researcher
concludes that the effect of treatment is not likely due
to chance.
94

(Portney and Watkins, 2000)

Inferential Statistics
Hypothesis Testing
Hypothesis Testing, continued…
Alternative Hypothesis: Any observed difference is
not due to chance.
Often the researcher is trying to support the
alternative hypothesis, as when trying to prove that
one particular treatment is better.
Sometimes, however, the researcher may be trying
to prove that certain interventions are equal.
95

(Portney and Watkins 2000)

Inferential Statistics
Hypothesis Testing, Errors
Errors in Hypothesis Testing
• Decision to accept or reject the null hypothesis
is based on the results of the statistical
procedures on collected data from samples.
• Decisions are based on sample data, so it is
possible that the results obtained are not accurate
of population data.
• There is a chance for error, that the researcher
may incorrectly accept or reject the null
hypothesis
96

(Portney and Watkins 2000)

Inferential Statistics
Hypothesis Testing, Errors
Type I error (α): Rejecting the null hypothesis when it
is true (for example, deciding that the difference seen
between a treatment group and a control group is
due to the effect of the treatment, when, in fact, the
difference is due to chance).
•A commonly used standard is α= 0.05, or the
researchers accept a 5% chance of making a
Type I error
Statistical tests completed with the sample data are
used to calculate p, the probability that an observed
difference did occur by chance.
97
(Portney and Watkins, 2000)

Inferential Statistics
Hypothesis Testing, p and α
Hypothesis Testing: Relationship between p and α
If p is greater than the chosen α, then the
researchers chose not to reject the null hypothesis
For example, in a placebo versus treatment study,
the researchers cannot conclude that the
experimental treatment had a different effect then the
placebo.
98

(Portney and Watkins, 2000)

Inferential Statistics
Hypothesis Testing, p and α
Hypothesis Testing: Relationship between p and α,
continued…
If p is less than the chosen α, then the researchers
chose to reject the null hypothesis
For example, in a placebo versus treatment study,
the researchers conclude that the experimental
treatment had a different effect then the placebo.
99

(Portney and Watkins, 2000)

Inferential Statistics
Hypothesis Testing, p and α
Hypothesis Testing: Relationship between p and α,
continued…
Confidence intervals surrounding the p value can be
calculated, hopefully these are included in data
analysis section of the research study.
The CIs between two groups should not (rare
exceptions) overlap, if a statistically significant
difference is found.
100

(Portney and Watkins, 2000)

Inferential Statistics
Hypothesis, CIs versus MCID
Hypothesis Testing : CIs (slide 90) versus MCID
(slide 67)
When the null hypothesis is rejected, the researchers
conclude that the experimental treatment (versus a
placebo, for example) had a statistically significant
effect.
The clinician should examine the effect size and the
MCID to ensure that the change is clinically and
functionally relevant.
101

(Guyatt and Rennie, 2002)

Inferential Statistics
Hypothesis, CIs versus MCID
When a null hypothesis is not rejected, the
researchers may conclude that the experimental
treatment did not have an effect.
However, the researchers should pay close attention
to the confidence intervals.
• If the CI does not include the MCID, then the trial
is most likely negative.
• If the CI includes the MCID, then the possibility
that the experimental treatment may have a
positive effect cannot be ruled out. The
researchers may wish to run a power analysis,
102
explained in later slides.
(Guyatt and Rennie, 2002)

Inferential Statistics, Hypothesis Testing
Clinical Example
Children with similar abilities/diagnosis
(homogenous sample) are randomly assigned to two
different groups:
• Group A receives a physical therapy designed to
improve gross motor skills, and
• Group B completes typical daily activities (but don’t
worry, due to ethical concerns children in Group B will
receive the same treatment as those in Group A at the
end of the study period).

103

Inferential Statistics, Hypothesis Testing
Clinical Example 1, continued…
The outcome measure will be a tool that tests gross
motor activities with a final, numerical outcome
score.
The authors hypothesis that the experimental group
will show statistically significant gains (experimental
hypothesis) and that the null hypothesis (there is no
difference between groups at the end of the
intervention period) will be rejected.
104

Inferential Statistics, Hypothesis Testing
Clinical Example 1, continued…
Initially, group A and B have similar average pre-test
scores (if not, that can be statistically corrected).
Now suppose that Group A (experimental group,
receiving additional PT services) increases an
average of 9 points from pretest to posttest, with a
CI of ±1, 8 to 10.
Group B increases an average of 0.5 points, with a
CI of ±2, -1.5 to 3.5.
105

Inferential Statistics, Hypothesis Testing
Clinical Example 1, continued…
These confidence intervals do not overlap, which
corresponds with the statistical analysis performed
that p