2017 Metopen Sesi 2 YM Riset Implementasi
Introduction to Implementation research
Yodi Mahendradhata
Key questions
• What do we mean by implementation research?
• Why do we need implementation research?
• How to formulate implementation research objectives and questions?
• What constructs/variables are relevant for implementation research?
• What are common methods for implementation research?
• What are the key references for implementation research?
What do e
Health system research?
ea
y …….
Implementation research?
Health policy research?
Operational research?
Health services research?
I ple e tatio resear h is that subset
of HSR that focuses on how to promote
the uptake and successful
implementation of evidence-based
interventions and policies….
(Sanders & Haines 2006)
…..usefully defi ed as s ie tifi inquiry
into questions concerning
implementation - the act of fulfilling or
carrying out an i te tio
(Peters et al 2013)
The ultiple defi itio s of operatio al
research, implementation research, and
health systems research creates confusion
and negatively affects the credibility
and progress of the resear h…
(Remme et al 2010)
Defining research to improve health systems
(Remme et al 2010)
Research Domain Primary characteristics
Focus
Users
Utility
Operational
Operational issues of specific
health programmes
Health care providers,
programme managers
Local
Implementation
Implementation strategies for
specific products or services
Programme managers, R&D
managers
Local/Broad
Health system
Issues affecting some or all of
the building blocks of a health
system
Health system managers,
policy makers
Broad
Defining research to improve health systems
(Remme et al 2010)
Examples of research questions for the three
research domains (Remme et al 2010)
Domain
Question
Operational
Which locations should be targeted for delivering HIV prevention services in Kawempe district,
Uganda?
Which of the current ART payment strategies in use in Nairobi should be retained for the new
integrated programme?
Implementation
How to improve access to vaccination among children who are currently not reached by
immunisation services?
How to deliver ivermectin for onchocerciasis control and ensure sustained high treatment
coverage in isolated rural communities?
Health system
How effective are different policies for attracting nurses to rural areas?
What has been the impact of the rapid scale-up of HIV programmes on fragile health systems?
Common interpretations of IR*
• Systematic approach to understanding and addressing barriers to
effective and quality implementation of health interventions,
strategies and policies.
• IR is demand-driven and the research questions are framed based on
needs identified together with relevant stakeholders/implementers in
the health system
*WHO (2014)
I ple e tatio resear h…*
• addresses implementation bottlenecks,
• identifies optimal approaches for a particular setting,
• promotes the uptake of research findings
• leads to improved health care and its delivery.
*WHO (2014)
Characteristics of Implementation Research*
*WHO (2014)
Systematic
Multidisciplinary
Contextual
Complex
Characteristics of Implementation Research (WHO 2013)
Systematic
•
•
The systematic study of how evidence-based public health interventions are integrated and
provided in specific settings, and how resulting health outcomes vary across communities.
Balances relevance to real life situations with rigor, strictly adhering to norms of scientific inquiry.
Multidisciplinary
• Analysis of biological, social, economic, political, system and environmental factors that impact
implementation of specific health interventions.
• Interdisciplinary collaborations between behavioural and social scientists, clinicians,
epidemiologists, statisticians, engineers, business analysts, policy makers, and key stakeholders.
Contextual
• Demand driven. Framing of research questions is based on needs identified by implementers in
the health system.
• Research is relevant to local specifics and needs, and aims to improve health care delivery in a
given context.
• Generates generalizable knowledge and insights that can be applied across various settings.
• Mindful of cultural and community-based influences.
Complex
•
•
•
•
Dynamic and adaptive.
Multi-scale: occurs at multiple levels of health systems and communities.
Analyses multi-component programmes and policies.
Non-linear, iterative, evolving process.
Why do we need implementation research?
E e he i ter e tio s are desig ed
in similar ways, implementation occurs
differently in different contexts, and
ith a y differe t effe ts.….
(WHO 2014)
The importance of implementation (WHO 2014)
Efficacious tool
Accessibility
Provider
compliance
Patient
adherence
Effectiveness
The Prevention of Mother-To-Child-Transmission
(PMTCT) cascade in Zambia (2007-2008)*
*Ref: Mandala et al., 2009
There is growing funding for intervention
research, particularly for drugs,
a i es….. Ho e er, there is still little
funding for, and, therefore, a relative
dearth of implementation resear h….
(Sanders & Haines 2006)
“o iety’s huge i est e t in technological
innovations that only modestly improve efficacy,
by consuming resources needed for improved
delivery of care, may cost more lives than it saves.
……Health, economic, and moral arguments make
the case for spending less on technological
advances and more on improving systems for
delivering care.
(Woolf & Johnson 2005)
Neglecting implementation
challenges costs lives and
money….
(WHO 2014)
Implementation research objectives and questions?
Implementation research objectives and questions*
Research objective
Illustration of research question
Explore
What are the possible factors and agents responsible for good
implementation of a health intervention? For enhancing or expanding a
health intervention
Describe
What describes the context in which implementation occurs? What describes
the main factors influencing implementation in a given context?
Influence
Is coverage of a health intervention changing among beneficiaries of the
intervention?
Explain
How and why does implementation of the intervention lead to effects on
health behaviour, services, or status in all its variations?
Predict
What is the likely course of future implementation?
*Peters et al 2013
IR Traditions, Research Questions, and Initial
Core Disciplines (Peters et al 2013)
Implementation Research
Tradition
Typical research questions
Core disciplines at
origin
Dissemination and
implementation of evidencebased Medicine
What promotes the integration of research findings and
evidence on interventions into healthcare policy and practice?
Epidemiology,
behavior change
(psychology,
sociology, education)
Participatory action
research
How can we (community members as research participants)
learn and be empowered to take action?
Social psychology,
education, and
anthropology
Management improvement
How are the right services delivered to the right clients while
meeting the right standards for quality?
Management,
engineering
Operational research
Which solution provides the most rational basis for a decision
concerning the optimal performance of a system?
Management,
engineering, math
Policy implementation
• How was a policy implemented, and what contributed to
reaching the objectives or other effects?
• Which actors are involved in programme delivery in specific
locations, how do they understand the problem of
implementation, and what influences their behaviour?
Political science,
public policy, public
administration
Constructs/variables for implementation research?
(proliferating) theoretical frameworks for IR?
• Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice:
models for dissemination and implementation research. American Journal of Preventive Medicine, 43(3),
337-350. doi: 10.1016/j.amepre.2012.05.024
• Flottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M., . . . Eccles, M. P.
(2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks
and taxonomies of factors that prevent or enable improvements in healthcare professional practice.
Implement Sci, 8, 35. doi: 10.1186/1748-5908-8-35
• Meyers, D. C., Durlak, J. A., & Wandersman, A. (2012). The quality implementation framework: a synthesis of
critical steps in the implementation process. American Journal of Community Psychology, 50(3-4), 462- 480.
doi: 10.1007/s10464-012-9522-x
• Cane, J., O'Connor, D., & Michie, S. (2012). Validation of the theoretical domains framework for use in
behaviour change and implementation research. Implement Sci, 7(1), 37. doi: 10.1186/1748-5908-7-37
• Hendriks, A.-M., Jansen, M., Gubbels, J., De Vries, N., Paulussen, T., & Kremers, S. (2013). Proposing a
conceptual framework for integrated local public health policy, applied to childhood obesity - the behavior
change ball. Implementation Science, 8(1), 46.
• Chaudoir S.R., et al. Measuring factors affecting implementation of health innovations: A systematic review
of structural, organizational, provider, patient, and innovation level measures. Implementation Science 2013;
8:22
• A d ore…..
Consolidated Framework for
Implementation Research (CFIR)*
Inner setting
Outer setting
Intervention
Individual
*Source: http://cfirguide.org/constructs.html
Process
Consolidated Framework for
Implementation Research (CFIR)*
Intervention
Outer setting
Inner setting
Individuals
Intervention source
Patient needs &
resources
Structural
characteristics
Knowledge & beliefs Planning
Evidence strength &
quality
Cosmopolitanism
Networks &
communications
Self-efficacy
Engaging
Relative advantage
Peer pressure
Culture
Individual stage of
change
Executing
Adaptability
External policy &
incentive
Implementation
climate
Identification with
organization
Reflecting &
evaluating
Readiness for
implementation
Other personal
attributes
Trialability
Complexity
Design quality &
packaging
Cost
*Source: http://cfirguide.org/constructs.html
Process
Implementation outcome variables
(Peters et al 2013; WHO 2014)
Variables
Working definition
Acceptability
The perception among stakeholders (for example, consumers, providers, managers, policy
makers) that an intervention is agreeable
Adoption
The intention, initial decision, or action to try to employ a new intervention
Appropriateness
The perceived fit or relevance of the intervention in a particular setting or for a particular
target audience (for example, provider or consumer) or problem
Feasibility
The extent to which an intervention can be carried out in a particular setting or
organization
Fidelity
The degree to which an intervention was implemented as it was designed in an original
protocol, plan, or policy
Implementation
cost
The incremental cost of the implementation strategy (for example, how the services are
delivered in a particular setting). The total cost of implementation would also include the
cost of the intervention itself
Coverage
The degree to which the population that is eligible to benefit from an intervention actually
receives it.
Sustainability
The extent to which an intervention is maintained or institutionalized in a given setting
Common methods for Implementation research?
Methods particularly suitable for IR
(Peters et al 2013)
Mixed methods
Participatory
action research
Effectivenessimplementation
hybrid trials
Quality
improvement
studies
Pragmatic trials
Differentiating Implementation research?
Spectrum of implementation research relevance
(Peters et al 2013)
Implementation
not relevant
Implementation
relevant but not
considered
Implementation Implementation Implementation
relevant but
studied as
as primary focus
effects reduced contributing
factors
Context
Controlled
Largely controlled Real world &
partial control
Real world &
population
Real world &
population
IR Variables
Not relevant
Assumed to be
controlled or not
relevant
Controlled
Independent
variables
Primary
outcome/
determinants
Examples
Basic science,
Phase I and II
clinical trials
Efficacy study,
Phase III RCT
Pragmatic trials,
Quasi
experimental
study with
comparison
Participatory
research,
Observational
studies
assessing
implementation
Mixed method
to determine
acceptability
Key questions to assess research designs or
reports on implementation research (Peters et al 2013)
• Does the research clearly aim to answer a question concerning implementation?
• Does the research clearly identify the primary audiences for the research and how they
would use the research?
• Is there a clear description of what is being implemented (for example, details of the
practice, programme, or policy)?
• Does the research involve an implementation strategy? If so, is it described and
examined in its fullness?
• Is the research co ducted i a real world setti g? If so, is the co text a d sa ple
population described in sufficient detail?
• Does the research appropriately consider implementation outcome variables?
• Does the research appropriately consider context and other factors that influence
implementation?
• Does the research appropriately consider changes over time and the level of complexity
of the system, including unintended consequences?
Key references?
Discussion time!
On-going developments
• Postgraduate scholarships for Implementation research
• Implementation research workshops
• Implementation research Massive Open Online Course
• Ethics in implementation research
• Reporting guidelines for implementation research
• Implementation research grants
• Updating of implementation research toolkit
• National strategy for implementation research
Yodi_Mahendradhata@yahoo.co.uk
Yodi Mahendradhata
Key questions
• What do we mean by implementation research?
• Why do we need implementation research?
• How to formulate implementation research objectives and questions?
• What constructs/variables are relevant for implementation research?
• What are common methods for implementation research?
• What are the key references for implementation research?
What do e
Health system research?
ea
y …….
Implementation research?
Health policy research?
Operational research?
Health services research?
I ple e tatio resear h is that subset
of HSR that focuses on how to promote
the uptake and successful
implementation of evidence-based
interventions and policies….
(Sanders & Haines 2006)
…..usefully defi ed as s ie tifi inquiry
into questions concerning
implementation - the act of fulfilling or
carrying out an i te tio
(Peters et al 2013)
The ultiple defi itio s of operatio al
research, implementation research, and
health systems research creates confusion
and negatively affects the credibility
and progress of the resear h…
(Remme et al 2010)
Defining research to improve health systems
(Remme et al 2010)
Research Domain Primary characteristics
Focus
Users
Utility
Operational
Operational issues of specific
health programmes
Health care providers,
programme managers
Local
Implementation
Implementation strategies for
specific products or services
Programme managers, R&D
managers
Local/Broad
Health system
Issues affecting some or all of
the building blocks of a health
system
Health system managers,
policy makers
Broad
Defining research to improve health systems
(Remme et al 2010)
Examples of research questions for the three
research domains (Remme et al 2010)
Domain
Question
Operational
Which locations should be targeted for delivering HIV prevention services in Kawempe district,
Uganda?
Which of the current ART payment strategies in use in Nairobi should be retained for the new
integrated programme?
Implementation
How to improve access to vaccination among children who are currently not reached by
immunisation services?
How to deliver ivermectin for onchocerciasis control and ensure sustained high treatment
coverage in isolated rural communities?
Health system
How effective are different policies for attracting nurses to rural areas?
What has been the impact of the rapid scale-up of HIV programmes on fragile health systems?
Common interpretations of IR*
• Systematic approach to understanding and addressing barriers to
effective and quality implementation of health interventions,
strategies and policies.
• IR is demand-driven and the research questions are framed based on
needs identified together with relevant stakeholders/implementers in
the health system
*WHO (2014)
I ple e tatio resear h…*
• addresses implementation bottlenecks,
• identifies optimal approaches for a particular setting,
• promotes the uptake of research findings
• leads to improved health care and its delivery.
*WHO (2014)
Characteristics of Implementation Research*
*WHO (2014)
Systematic
Multidisciplinary
Contextual
Complex
Characteristics of Implementation Research (WHO 2013)
Systematic
•
•
The systematic study of how evidence-based public health interventions are integrated and
provided in specific settings, and how resulting health outcomes vary across communities.
Balances relevance to real life situations with rigor, strictly adhering to norms of scientific inquiry.
Multidisciplinary
• Analysis of biological, social, economic, political, system and environmental factors that impact
implementation of specific health interventions.
• Interdisciplinary collaborations between behavioural and social scientists, clinicians,
epidemiologists, statisticians, engineers, business analysts, policy makers, and key stakeholders.
Contextual
• Demand driven. Framing of research questions is based on needs identified by implementers in
the health system.
• Research is relevant to local specifics and needs, and aims to improve health care delivery in a
given context.
• Generates generalizable knowledge and insights that can be applied across various settings.
• Mindful of cultural and community-based influences.
Complex
•
•
•
•
Dynamic and adaptive.
Multi-scale: occurs at multiple levels of health systems and communities.
Analyses multi-component programmes and policies.
Non-linear, iterative, evolving process.
Why do we need implementation research?
E e he i ter e tio s are desig ed
in similar ways, implementation occurs
differently in different contexts, and
ith a y differe t effe ts.….
(WHO 2014)
The importance of implementation (WHO 2014)
Efficacious tool
Accessibility
Provider
compliance
Patient
adherence
Effectiveness
The Prevention of Mother-To-Child-Transmission
(PMTCT) cascade in Zambia (2007-2008)*
*Ref: Mandala et al., 2009
There is growing funding for intervention
research, particularly for drugs,
a i es….. Ho e er, there is still little
funding for, and, therefore, a relative
dearth of implementation resear h….
(Sanders & Haines 2006)
“o iety’s huge i est e t in technological
innovations that only modestly improve efficacy,
by consuming resources needed for improved
delivery of care, may cost more lives than it saves.
……Health, economic, and moral arguments make
the case for spending less on technological
advances and more on improving systems for
delivering care.
(Woolf & Johnson 2005)
Neglecting implementation
challenges costs lives and
money….
(WHO 2014)
Implementation research objectives and questions?
Implementation research objectives and questions*
Research objective
Illustration of research question
Explore
What are the possible factors and agents responsible for good
implementation of a health intervention? For enhancing or expanding a
health intervention
Describe
What describes the context in which implementation occurs? What describes
the main factors influencing implementation in a given context?
Influence
Is coverage of a health intervention changing among beneficiaries of the
intervention?
Explain
How and why does implementation of the intervention lead to effects on
health behaviour, services, or status in all its variations?
Predict
What is the likely course of future implementation?
*Peters et al 2013
IR Traditions, Research Questions, and Initial
Core Disciplines (Peters et al 2013)
Implementation Research
Tradition
Typical research questions
Core disciplines at
origin
Dissemination and
implementation of evidencebased Medicine
What promotes the integration of research findings and
evidence on interventions into healthcare policy and practice?
Epidemiology,
behavior change
(psychology,
sociology, education)
Participatory action
research
How can we (community members as research participants)
learn and be empowered to take action?
Social psychology,
education, and
anthropology
Management improvement
How are the right services delivered to the right clients while
meeting the right standards for quality?
Management,
engineering
Operational research
Which solution provides the most rational basis for a decision
concerning the optimal performance of a system?
Management,
engineering, math
Policy implementation
• How was a policy implemented, and what contributed to
reaching the objectives or other effects?
• Which actors are involved in programme delivery in specific
locations, how do they understand the problem of
implementation, and what influences their behaviour?
Political science,
public policy, public
administration
Constructs/variables for implementation research?
(proliferating) theoretical frameworks for IR?
• Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice:
models for dissemination and implementation research. American Journal of Preventive Medicine, 43(3),
337-350. doi: 10.1016/j.amepre.2012.05.024
• Flottorp, S. A., Oxman, A. D., Krause, J., Musila, N. R., Wensing, M., Godycki-Cwirko, M., . . . Eccles, M. P.
(2013). A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks
and taxonomies of factors that prevent or enable improvements in healthcare professional practice.
Implement Sci, 8, 35. doi: 10.1186/1748-5908-8-35
• Meyers, D. C., Durlak, J. A., & Wandersman, A. (2012). The quality implementation framework: a synthesis of
critical steps in the implementation process. American Journal of Community Psychology, 50(3-4), 462- 480.
doi: 10.1007/s10464-012-9522-x
• Cane, J., O'Connor, D., & Michie, S. (2012). Validation of the theoretical domains framework for use in
behaviour change and implementation research. Implement Sci, 7(1), 37. doi: 10.1186/1748-5908-7-37
• Hendriks, A.-M., Jansen, M., Gubbels, J., De Vries, N., Paulussen, T., & Kremers, S. (2013). Proposing a
conceptual framework for integrated local public health policy, applied to childhood obesity - the behavior
change ball. Implementation Science, 8(1), 46.
• Chaudoir S.R., et al. Measuring factors affecting implementation of health innovations: A systematic review
of structural, organizational, provider, patient, and innovation level measures. Implementation Science 2013;
8:22
• A d ore…..
Consolidated Framework for
Implementation Research (CFIR)*
Inner setting
Outer setting
Intervention
Individual
*Source: http://cfirguide.org/constructs.html
Process
Consolidated Framework for
Implementation Research (CFIR)*
Intervention
Outer setting
Inner setting
Individuals
Intervention source
Patient needs &
resources
Structural
characteristics
Knowledge & beliefs Planning
Evidence strength &
quality
Cosmopolitanism
Networks &
communications
Self-efficacy
Engaging
Relative advantage
Peer pressure
Culture
Individual stage of
change
Executing
Adaptability
External policy &
incentive
Implementation
climate
Identification with
organization
Reflecting &
evaluating
Readiness for
implementation
Other personal
attributes
Trialability
Complexity
Design quality &
packaging
Cost
*Source: http://cfirguide.org/constructs.html
Process
Implementation outcome variables
(Peters et al 2013; WHO 2014)
Variables
Working definition
Acceptability
The perception among stakeholders (for example, consumers, providers, managers, policy
makers) that an intervention is agreeable
Adoption
The intention, initial decision, or action to try to employ a new intervention
Appropriateness
The perceived fit or relevance of the intervention in a particular setting or for a particular
target audience (for example, provider or consumer) or problem
Feasibility
The extent to which an intervention can be carried out in a particular setting or
organization
Fidelity
The degree to which an intervention was implemented as it was designed in an original
protocol, plan, or policy
Implementation
cost
The incremental cost of the implementation strategy (for example, how the services are
delivered in a particular setting). The total cost of implementation would also include the
cost of the intervention itself
Coverage
The degree to which the population that is eligible to benefit from an intervention actually
receives it.
Sustainability
The extent to which an intervention is maintained or institutionalized in a given setting
Common methods for Implementation research?
Methods particularly suitable for IR
(Peters et al 2013)
Mixed methods
Participatory
action research
Effectivenessimplementation
hybrid trials
Quality
improvement
studies
Pragmatic trials
Differentiating Implementation research?
Spectrum of implementation research relevance
(Peters et al 2013)
Implementation
not relevant
Implementation
relevant but not
considered
Implementation Implementation Implementation
relevant but
studied as
as primary focus
effects reduced contributing
factors
Context
Controlled
Largely controlled Real world &
partial control
Real world &
population
Real world &
population
IR Variables
Not relevant
Assumed to be
controlled or not
relevant
Controlled
Independent
variables
Primary
outcome/
determinants
Examples
Basic science,
Phase I and II
clinical trials
Efficacy study,
Phase III RCT
Pragmatic trials,
Quasi
experimental
study with
comparison
Participatory
research,
Observational
studies
assessing
implementation
Mixed method
to determine
acceptability
Key questions to assess research designs or
reports on implementation research (Peters et al 2013)
• Does the research clearly aim to answer a question concerning implementation?
• Does the research clearly identify the primary audiences for the research and how they
would use the research?
• Is there a clear description of what is being implemented (for example, details of the
practice, programme, or policy)?
• Does the research involve an implementation strategy? If so, is it described and
examined in its fullness?
• Is the research co ducted i a real world setti g? If so, is the co text a d sa ple
population described in sufficient detail?
• Does the research appropriately consider implementation outcome variables?
• Does the research appropriately consider context and other factors that influence
implementation?
• Does the research appropriately consider changes over time and the level of complexity
of the system, including unintended consequences?
Key references?
Discussion time!
On-going developments
• Postgraduate scholarships for Implementation research
• Implementation research workshops
• Implementation research Massive Open Online Course
• Ethics in implementation research
• Reporting guidelines for implementation research
• Implementation research grants
• Updating of implementation research toolkit
• National strategy for implementation research
Yodi_Mahendradhata@yahoo.co.uk