Week 12 questionnaire and instruments

  Chapter 13 Chapter 13 Questionnaires Questionnaires and and

  Instruments Instruments

  Learning Objectives Learning Objectives Understand...

• The link forged between the management

  

dilemma and the communication instrument by

the management-research question hierarchy.

• The influence of the communication method on instrument design.
• The three general classes of information and what each contributes to the instrument.

  Learning Objectives Learning Objectives Understand . . .

• The influence of question content, question

  wording, response strategy, and preliminary analysis planning on question construction.

• Each of the numerous question design issues

  influencing instrument quality, reliability, and validity.

• The sources for measurement questions
• The importance of pretesting questions and instruments.

  Measurement Skepticism Measurement Skepticism

  “Research that asks consumers what they did and why is incredibly helpful.

  Research that asks consumers what they are going to do can often be taken with a grain of salt.”

  Al Ries author, co-founder, and chairman Ries & Ries.

  PulsePoint: PulsePoint: Research Revelation Research Revelation

  Overall Flowchart for Instrument Overall Flowchart for Instrument Design Design

  Flowchart for Instrument Flowchart for Instrument Design Phase 1 Design Phase 1

  Strategic Concerns in Strategic Concerns in Instrument Design Instrument Design

  

What type of scale is needed?

What communication approach will be used?

Should the questions be structured?

Should the questioning be disguised?

  Technology Affects Technology Affects Questionnaire Development Questionnaire Development WebSurveyor used to write an instrument.

  Write questionnaires more quickly Create visually driven instruments

  Eliminate manual data entry Save time in data analysis

  Disguising Study Objectives Disguising Study Objectives Reluctantly shared,

  Willingly shared, Conscious-level

  Conscious-level information information

  Situations Situations where where disguise is disguise is unnecessary unnecessary

  Knowable, Subconscious-level

  Limited-conscious- information level information

  Dummy Table for Dummy Table for American Eating Habits American Eating Habits

  Use of Convenience Foods

_{Age Use Frequently s Rarely Use Never Use}

_{Always Use Sometime} Use _25-34 18-24 _55-64 35-44 65+

  Flowchart for Instrument Design Flowchart for Instrument Design Phase 2 Phase 2

  Question Categories Question Categories and Structure and Structure

  Administrative Target Classification

  Engagement = Convenience Engagement = Convenience

  “Participants are becoming more and more aware of the value of their time. The key to maintaining a quality dialog with them is to make it really convenient for them to engage, whenever and wherever they want.”

  Tom Anderson managing partner Anderson Analytics

  Question Content Question Content

  Should this question be asked? Is the question of proper scope and coverage? Can the participant adequately answer this question as asked? Will the participant willingly answer this question as asked?

  Question Wording Question Wording

  Shared vocabulary Adequate Single alternatives meaning Criteria Criteria Misleading Personalized assumptions Biased

  Response Strategy Response Strategy

  Factors Factors

  Objectives of the study Participant’s level of information

  Degree to which participants have thought through topic

  Ease and clarity with which participant communicates

  Participant’s motivation to share

  Free-Response Strategy Free-Response Strategy

  What factors influenced your enrollment in Metro U? ____________________________________________ ____________________________________________

  Dichotomous Response Dichotomous Response Strategy Strategy

  

Did you attend the “A Day at College” program at Metro U?

  Yes No

  Multiple Choice Response Multiple Choice Response Strategy Strategy

Which one of the following factors was most influential

_{in your decision to attend Metro U? Enjoyable campus life Specific program of study desired} Good academic standing Many friends from home

  Checklist Response Strategy Checklist Response Strategy _{your decision to enroll in Metro U? (Check all that apply.)} Which of the following factors influenced _{ Parents’ preferences  Specific program of study desired}  Tuition cost _{ High quality of faculty}

_{ Many friends from home attend}

 Opinion of brother or sister

  Rating Response Strategy Rating Response Strategy

  

Strongly

influential

Somewhat influential

  Not at all influential _{Good academic reputation   } Enjoyable campus life   

Many friends   

  High quality faculty _{  } Semester calendar   

  Ranking Ranking _{Use 1 to indicate the most encouraging factor, 2 the next most based on their influence in encouraging you to apply to Metro U.} Please rank-order your top three factors from the following list _{_____ Closeness to home _____ Opportunity to play collegiate sports} encouraging factor, etc. _{_____ High quality of faculty _____ Good academic reputation} _____ Enjoyable campus life

  Summary of Scale Types Summary of Scale Types

  Type Restrictions Scale Data Type _{Rating Scales Items} Simple Category _{Multiple Choice Scale • Needs mutually exclusive choices Many Nominal • Needs mutually exclusive choices One or Nominal more} Single-Response _{Scale • May use exhaustive list or ‘other’ Multiple-Response} Multiple Choice _{(checklist) Scale • Needs exhaustive list or ‘other’ • Needs mutually exclusive choices Many Nominal} Likert Scale

• _{• Needs definitive positive or One or Ordinal agree/disagree negative statements with which to more} Likert-type Scale
• _{•Needs definitive positive or One or Ordinal negative statements with which to more}

  Summary of Scale Types Summary of Scale Types

  

Type Restrictions Scale Data Type

_{Rating Scales Items} Numerical _{Scale •Score is a measurement of graphical space •Needs number anchors of the scale or end-points many •Needs concepts with standardized meanings; One or} ^{Ordinal or} _{Interval Rating List} Multiple _{Scale end-points on the verbal scale •Needs words that are opposites to anchor the Up to 10} ^Ordinal Fixed Sum _{Scale some fixed number, often 100. more •Participant needs ability to calculate total to Two or} ^{Interval or Ratio}

  Summary of Scale Types Summary of Scale Types

  

Type Restrictions Scale Data Type

_{Items Stapel •Needs verbal labels that are operationally One or} Rating Scales _{Ordinal or Graphic} Scale _{defined or standard. •Needs visual images that can be interpreted as One or more Ordinal} ^Interval Rating _{Scale from one anchor. •Score is a measurement of graphical space positive or negative anchors more} ^{(Interval, or} _Ratio)

  Summary of Scale Types Summary of Scale Types

  

Type Restrictions Scale Data Type

_{Items Comparison stamina and interest.}

_{Paired • Number is controlled by participant’s Up to 10 Ordinal}

Ranking Scales _{Forced Ranking • Needs mutually exclusive choices. Up to 10 Ordinal or} Scale _{Comparative • Can use verbal or graphical scale. Up to 10 Ordinal} Scale ^Interval Scale

  Internet Survey Internet Survey Scale Options Scale Options

  Internet Survey Internet Survey Scale Options Scale Options

  Internet Survey Internet Survey Scale Options Scale Options

  Sources of Questions Sources of Questions

• Handbook of Marketing
• Index to International
• The Gallup Poll
• Sourcebook of Harris
• Measures of
• Marketing Scales
• American Social
• Measures of Political

  Scales

  Cumulative Index

  Personality and Social- Psychological Attitudes

  Attitudes

  Public Opinion

  National Surveys

  Handbook

  Attitudes Data Sourcebook

  Flowchart for Instrument Flowchart for Instrument Design Phase 3 Design Phase 3

  Guidelines for Guidelines for Question Sequencing Question Sequencing Interesting topics early

  Interesting topics early Simple topics early

  Simple topics early Sensitive questions later

  Sensitive questions later Classification questions later

  Classification questions later Transition between topics

  Transition between topics Reference changes limited

  Reference changes limited

  Illustrating the Funnel Illustrating the Funnel Approach Approach

  How do you think this country is getting along in its

relations with other countries?

How do you think we are doing in our relations with

Iran?

  Do you think we ought to be dealing with Iran

differently than we are now?

(If yes) What should we be doing differently? Some people say we should get tougher with Iran

and others think we are too tough as it is; how

do you feel about it?

  Branching Question Branching Question

  Components of Components of Questionnaires Questionnaires

  MindWriter Survey MindWriter Survey

  Overcoming Instrument Overcoming Instrument Problems Problems Build rapport

  Build rapport Redesign question process

  Redesign question process Explore alternatives

  Explore alternatives Use other methods

  Use other methods Pretest

  Pretest

  Key Terms Key Terms

• Administrative
• Disguised question
• Double-barreled

  question

  question

• Branched question
• Buffer question
• Checklist • Classification question
• Dichotomous question>Free-response question
• Interview schedule
• Leading question
• Multiple-choice question

  Key Terms Key Terms

• Pretesting • Screen question
• Primacy effect • Structured response
• Ranking question • Target question
• – Structured
• Rating question
• – Unstructured